Merge pull request #4 from intel/develop

Migrate OpenFL 1.0 to GitHub

.gitignore

@@ -0,0 +1,6 @@
+*.egg-info
+__pycache__
+/build
+/dist
+.vscode
+.ipynb_checkpoints

MANIFEST.in

@@ -0,0 +1,3 @@
+recursive-include openfl-workspace *
+recursive-include openfl-docker *
+recursive-include openfl-tutorials *

NOTICE

@@ -0,0 +1,15 @@
+from gprc:
+
+Copyright 2014 gRPC authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

README.md

@@ -1,3 +1,73 @@
+
 # Welcome to Intel® OpenFL
 
-Official release coming soon. See the `develop` branch for more information. 
+[Federated learning](https://en.wikipedia.org/wiki/Federated_learning) is a distributed machine learning approach that
+enables organizations to collaborate on machine learning projects
+without sharing sensitive data, such as, patient records, financial data,
+or classified secrets 
+([Sheller MJ,  et al., 2020](https://www.nature.com/articles/s41598-020-69250-1);
+[Sheller MJ, et al., 2019](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6589345);
+[Yang Y, et al., 2019](https://arxiv.org/abs/1902.04885);
+[McMahan HB, et al., 2016](https://arxiv.org/abs/1602.05629)).
+
+
+The basic premise behind federated learning
+is that the model moves to meet the data rather than the data moving
+to meet the model. Therefore, the minimum data movement needed
+across the federation is solely the model parameters and their updates.
+
+
+OpenFL is a Python 3 project developed by Intel Labs and 
+Intel Internet of Things Group. 
+
+![Federated Learning](docs/images/diagram_fl.png)
+
+## Requirements
+
+- OS: Tested on Ubuntu Linux 16.04 and 18.04.
+- Python 3.6+ with a Python virtual environment (e.g. [conda](https://docs.conda.io/en/latest/))
+- TensorFlow 2+ or PyTorch 1.6+ (depending on your training requirements). OpenFL is designed to easily support other frameworks as well.
+
+![fx commandline interface](docs/images/fx_help.png)
+
+## License
+This project is licensed under [Apache License Version 2.0](LICENSE).
+By contributing to the project, you agree to the license and copyright terms therein
+and release your contribution under these terms.
+
+## Resources:
+* Docs: 
+* Wiki: 
+* Issue tracking: 
+
+## Support
+Please report questions, issues and suggestions using:
+
+* [GitHub* Issues]()
+
+### Relation to OpenFederatedLearning and the Federated Tumor Segmentation (FeTS) Initiative
+
+This project builds on the [Open Federated Learning](https://github.com/IntelLabs/OpenFederatedLearning) framework that was 
+developed as part of a collaboration between Intel
+and the University of Pennsylvania (UPenn) for federated learning. 
+It describes Intel’s commitment in 
+supporting the grant awarded to the [Center for Biomedical Image Computing and Analytics (CBICA)](https://www.cbica.upenn.edu/) 
+at UPenn (PI: S. Bakas) from the [Informatics Technology for Cancer Research (ITCR)](https://itcr.cancer.gov/) program of 
+the National Cancer Institute (NCI) of the National Institutes of Health (NIH), 
+for the development of the [Federated Tumor Segmentation (FeTS, www.fets.ai)](https://www.fets.ai/) 
+platform (grant award number: U01-CA242871). 
+
+FeTS is an exciting, real-world 
+medical FL platform, and we are honored to be collaborating with UPenn in 
+leading a federation of international collaborators. The original OpenFederatedLearning
+project and OpenFL are designed to serve as the backend for the FeTS platform, 
+and OpenFL developers and researchers continue to work very closely with UPenn on 
+the FeTS project. The 
+[FeTS-AI/Front-End](https://github.com/FETS-AI/Front-End) shows how UPenn 
+and Intel have integrated UPenn’s medical AI expertise with Intel’s framework 
+to create a federated learning solution for medical imaging. 
+
+Although initially developed for use in medical imaging, this project is
+designed to be agnostic to the use-case, the industry, and the 
+machine learning framework.
+

docs/.gitignore

@@ -0,0 +1,5 @@
+openfl*
+models*
+data*
+/_build
+**/.ipynb_checkpoints

docs/Makefile

@@ -0,0 +1,27 @@
+# Copyright (C) 2020-2021 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+
+	sphinx-apidoc -f -o . ../openfl
+
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/README.md

@@ -0,0 +1,29 @@
+# How to update the documentation
+
+We use sphinx to generate the documentation for this project.
+The documentation project has been initialized properly and we basically just need to update the actual content.
+
+If we ever change the code structure since last compilation, we may need to regenerate the docstring index:
+```shell
+sphinx-apidoc -f -o . ../openfl
+sphinx-apidoc -f -o . ../models
+```
+
+The command detects the code structure under `../openfl` and generates a series of `*.rst` files, such as `openfl.aggregator.rst`.
+However, the docstring would not be compiled until we execute `make html` later.
+
+We can also update the hand-crafted documents, including `intro.rst` and `tutorial.rst`. The `index.rst` is the entry file. We don't need to modify it unless we want to add more hand-crafted pages or adjust the order in the Contents page.
+
+
+After completing revision on the .rst files, we would compile the documentation source code:
+```
+make clean
+make html
+```
+
+The Makefile supports many targets. We choose html because we can easily host the documentation on a remote server:
+
+```shell
+cd _build/html
+python -m http.server
+```

docs/advanced_topics.rst

@@ -0,0 +1,15 @@
+.. # Copyright (C) 2020 Intel Corporation
+.. # Licensed subject to the terms of the separately executed evaluation license agreement between Intel Corporation and you.
+
+.. _advanced_topics:
+
+***************
+Advanced Topics
+***************
+
+.. toctree::
+   :maxdepth: 4
+
+   multiple_plans
+
+

docs/conf.py

@@ -0,0 +1,110 @@
+"""Docs configuration module."""
+
+# Copyright (C) 2020-2021 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+
+# Configuration file for the Sphinx documentation builder.
+#
+# This file only contains a selection of the most common options. For a full
+# list see the documentation:
+# http://www.sphinx-doc.org/en/master/config
+
+# -- Path setup --------------------------------------------------------------
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#
+import os
+import sys
+from datetime import datetime
+
+sys.path.insert(0, os.path.abspath('../'))
+
+# -- General configuration ---------------------------------------------------
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+# import sphinx_rtd_theme
+# import sphinxcontrib.napoleon
+
+extensions = [
+    "sphinx_rtd_theme",
+    'sphinx.ext.autosectionlabel',
+    "sphinx.ext.napoleon",
+    "sphinx-prompt",
+    'sphinx_substitution_extensions',
+    "sphinx.ext.ifconfig",
+    "sphinxcontrib.mermaid"
+]
+
+# -- Project information -----------------------------------------------------
+
+# This will replace the |variables| within the rST documents automatically
+
+PRODUCT_VERSION = "Intel"
+# PRODUCT_VERSION = "OFL"
+
+# tags.add(PRODUCT_VERSION)
+
+if PRODUCT_VERSION == "Intel":
+
+    project = 'OpenFL'
+    copyright = '{}, Intel'.format(datetime.now().year)
+    author = 'Intel Corporation'
+    version = "{}.{}".format(datetime.now().year, datetime.now().month)
+    release = version
+    master_doc = 'index'
+
+    # Global variables for rST
+    rst_prolog = """
+    .. |productName| replace:: OpenFL
+    .. |productZip| replace:: openfl.zip
+    .. |productDir| replace:: openfl
+    .. |productWheel| replace:: openfl
+
+    """
+
+else:
+
+    project = 'Open Federated Learning'
+    author = 'FeTS'
+    master_doc = 'index'
+    version = "{}.{}".format(datetime.now().year, datetime.now().month)
+    release = version
+
+    # Global variables for rST
+    rst_prolog = """
+    .. |productName| replace:: Open Federated Learning
+    .. |productZip| replace:: OpenFederatedLearning.zip
+    .. |productDir| replace:: OpenFederatedLearning
+    .. |productWheel| replace:: openfl
+
+    .. _Makefile: https://github.com/IntelLabs/OpenFederatedLearning/blob/master/Makefile
+    """
+
+napoleon_google_docstring = True
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+# This pattern also affects html_static_path and html_extra_path.
+exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', "tutorials/*", "graveyard/*"]
+
+# -- Options for HTML output -------------------------------------------------
+
+# The theme to use for HTML and HTML Help pages.  See the documentation for
+# a list of builtin themes.
+#
+html_theme = 'sphinx_rtd_theme'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+html_static_path = ['_static']
+
+autosectionlabel_prefix_document = True

docs/index.rst

@@ -0,0 +1,45 @@
+.. # Copyright (C) 2020-2021 Intel Corporation
+.. # SPDX-License-Identifier: Apache-2.0.
+
+
+.. Documentation master file, created by
+   sphinx-quickstart on Thu Oct 24 15:07:19 2019.
+   You can adapt this file completely to your liking, but it should at least
+   contain the root `toctree` directive.
+
+***************************************
+Welcome to the |productName| documentation!
+***************************************
+
+Intel\ :sup:`®` \ Federated Learning (|productName|) is a Python3 library for federated learning.
+It enables organizations to collaborately train a
+model without sharing sensitive information with each other.
+
+There are basically two components in the library:
+the *collaborator* which uses local dataset to train
+a global model and the *aggregator* which receives
+model updates from collaborators and combines them to form 
+the global model.
+
+The *aggregator* is framework-agnostic, while the *collaborator*
+can use any deep learning frameworks, such as `Tensorflow <https://www.tensorflow.org/>`_ or
+`PyTorch <https://pytorch.org/>`_.
+
+|productName| is developed by Intel Labs and Intel Internet of Things Group.
+
+.. toctree::
+   :maxdepth: 4
+   :caption: Contents:
+
+   manual
+   openfl
+   models
+   data
+
+
+Indices and tables
+==================
+
+* :ref:`genindex`
+* :ref:`modindex`
+* :ref:`search`

docs/install.design.rst

@@ -0,0 +1,17 @@
+.. # Copyright (C) 2020-2021 Intel Corporation
+.. # SPDX-License-Identifier: Apache-2.0
+
+*****************
+Design Philosophy
+*****************
+
+The overall design centers around the *Federated Learning (FL) Plan*.
+The plan is just a `YAML <https://en.wikipedia.org/wiki/YAML>`_
+file that defines the
+collaborators, aggregator, connections, models, data,
+and any other parameters that describes how the training will evolve.
+In the “Hello Federation” demos, the plan will be located in the
+YAML file within the federation project's workspace: *plan/plan.yaml*.
+As you modify federation workspace, you’ll effectively
+just be editing the plan along with the Python code defining
+the model and the data loader in order to meet your requirements.