Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Workflow Interface FedProx Example with Synthetic Dataset #734

Open
wants to merge 31 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
ab217e7
Adding Workflow Interface FedProx Example with Synthetic Dataset
ParthM-GitHub Feb 14, 2023
ba40762
Modified FedProx Example, tested following scenarios
ParthM-GitHub Feb 21, 2023
b56adfe
Uncommented few necessary lines
ParthM-GitHub Feb 21, 2023
bc3b19c
Updated FedProx example
ParthM-GitHub Feb 21, 2023
8fc8111
Updated FedProx example
ParthM-GitHub Feb 21, 2023
4eba802
Updated FedProx example
ParthM-GitHub Feb 21, 2023
29728d7
Update Workflow_Interface_401_FedProx_with_Synthetic_nonIID.ipynb
ParthM-GitHub Feb 22, 2023
2a3cc29
Changes as per numpy 1.24.x
ParthM-GitHub Feb 22, 2023
717a79b
Update README.md (#719)
grib0ed0v Feb 3, 2023
8f6e31a
Add files via upload
operepel Feb 9, 2023
936a707
Add files via upload
operepel Feb 9, 2023
f5afec2
Update CONTRIBUTING.md
operepel Feb 9, 2023
445dba4
Update CONTRIBUTING.md
operepel Feb 9, 2023
9e5b313
Update README.md
operepel Feb 9, 2023
bed9626
Fixed workflow interface notebook requirements (#729)
psfoley Feb 13, 2023
529e7e5
Update CONTRIBUTING.md
operepel Feb 14, 2023
f8fb971
Update CONTRIBUTING.md
operepel Feb 14, 2023
28c90a4
Fix CONTINUE_GLOBAL optimizer treatment (#711)
itrushkin Feb 14, 2023
013f4f1
Added following changes:
ParthM-GitHub Mar 1, 2023
cbd1568
Merge branch 'intel:develop' into fedprox_example
ParthM-GitHub Mar 1, 2023
2be8d78
Refactored WeightedAverage:
ParthM-GitHub Mar 9, 2023
051abf3
Added __init__.py file in tests/openfl/experimental folder.
ParthM-GitHub Mar 9, 2023
a9e2ca0
Fixing lint suggestions
ParthM-GitHub Mar 10, 2023
2346f4c
Fixing lint suggestions.
ParthM-GitHub Mar 10, 2023
9cba0ff
Fixing lint suggestions.
ParthM-GitHub Mar 10, 2023
6658023
Fixing lint suggestions.
ParthM-GitHub Mar 10, 2023
65e7b55
Adding optimizer.pth file for optimizer weighted average test case.
ParthM-GitHub Mar 10, 2023
ce024e0
Merge branch 'securefederatedai:develop' into fedprox_example
ParthM-GitHub Mar 24, 2023
b66b22f
Merge branch 'securefederatedai:develop' into fedprox_example
ParthM-GitHub Apr 10, 2023
6a7ab72
Merge branch 'securefederatedai:develop' into fedprox_example
ParthM-GitHub Jun 12, 2023
d98633d
Merge branch 'securefederatedai:develop' into fedprox_example
ParthM-GitHub Sep 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 101 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
# Contributing to OpenFL

We welcome contributions from the community. We believe that anyone can bring something valuable to OpenFL and help us to improve the project. This document explains how to contribute to OpenFL.

We accept various contributions from documentation improvement and bug fixing to major features proposals and [roadmap](https://github.com/intel/openfl/blob/develop/ROADMAP.md) suggestions.

Documentation improvement: review our [documentation](https://openfl.readthedocs.io/en/latest/install.html) and let us know if something is not clear or not relevant.
Propose your own formulations or even write new section explaining something that you know how works, but do not see in the documentation.
Propose it through GitHub [issues](https://github.com/intel/openfl/issues/new/choose) or [Discussions](https://github.com/intel/openfl/discussions).

To propose bugs, new features, or other code improvements:

1. Check open and closed [issues](https://github.com/intel/openfl/issues) and make sure there is no similar proposal.
2. Open a [new issue](https://github.com/intel/openfl/issues/new/choose), select a relevant category (Bug report / Feature request / Report a security vulnerability) and describe your idea using the template.
3. If you want to fix a bug or create this feature by yourself, prepare a contribution.
- Format your code following the [flake8 style](https://flake8.pycqa.org/en/latest/).
- Make sure that your code is original and corresponds to [OpenFL license](#license).
- Sing your work - [see below](#sign-your-work).
- Create a [pull request](#formatting-of-pull-requests) and wait for feedback.
- Verify that all tests in our [CI/CD pipeline](#Continuous-Integration-and-Continuous-Development) passed.
4. Hurrah! You are a new contributor to OpenFL! You will see your name in released notes of the subsequent releases!😊

Join our [Slack](https://join.slack.com/t/openfl/shared_invite/zt-ovzbohvn-T5fApk05~YS_iZhjJ5yaTw) and [Community meetings](https://github.com/intel/openfl#support) and participate in the discussions.

Are you an expert in Federated Learning and want to contribute to our roadmap? You can nominate yourself as a member of our Technical Steering Committee and be part of the OpenFL decision making group. Please reach us through our [Slack](https://join.slack.com/t/openfl/shared_invite/zt-ovzbohvn-T5fApk05~YS_iZhjJ5yaTw).

### Code format and style

We use [flake8](https://flake8.pycqa.org/en/latest/) for PEP8 style guide enforcement. This is run as a part of our CI/CD pipeline and it’s required prior a merge.

### Formatting of Pull Requests

OpenFL follows standard recommendations of PR formatting. Please find more details [here](https://github.blog/2015-01-21-how-to-write-the-perfect-pull-request/).

### Continuous Integration and Continuous Development

OpenFL uses GitHub actions to perform all functional and unit tests. Before your contribution can be merged make sure that all your tests are passing.
For more information of what fails you can click on the “details” link near the pipeline that failed.

![CI/CD](docs/images/CI_details.png)

### Writing the tests

The OpenFL team recommend including tests for all new features contributions. Test can be found in the “Tests” directory.
The [Tests/OpenFL folder](https://github.com/intel/openfl/tree/develop/tests/openfl) contains unit tests and the [Tests/GitHub folder](https://github.com/intel/openfl/tree/develop/tests/github) contains end-to-end and functional tests.

### License

OpenFL is licensed under the terms in [Apache 2.0 license](https://github.com/intel/openfl/blob/develop/LICENSE). By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

### Sign your work

Please use the sign-off line at the end of the patch. Your signature certifies that you wrote the patch or otherwise have the right to pass it on as an open-source patch. The rules are pretty simple: if you can certify
the below (from [developercertificate.org](http://developercertificate.org/)):

```
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
660 York Street, Suite 102,
San Francisco, CA 94110 USA

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.
```

Then you just add a line to every git commit message:

Signed-off-by: Joe Smith <[email protected]>

Use your real name (sorry, no pseudonyms or anonymous contributions.)

If you set your `user.name` and `user.email` git configs, you can sign your
commit automatically with `git commit -s`.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Open Federated Learning (OpenFL) - An Open-Source Framework For Federated Learning

[![PyPI - Python Version](https://img.shields.io/badge/python-3.6%20%7C%203.7%20%7C%203.8-blue)](https://pypi.org/project/openfl/)
[![PyPI - Python Version](https://img.shields.io/badge/python-3.7%20%7C%203.8%20%7C%203.9%20%7C%203.10-blue)](https://pypi.org/project/openfl/)
[![Ubuntu CI status](https://github.com/intel/openfl/actions/workflows/ubuntu.yml/badge.svg)](https://github.com/intel/openfl/actions/workflows/ubuntu.yml)
[![Windows CI status](https://github.com/intel/openfl/actions/workflows/windows.yml/badge.svg)](https://github.com/intel/openfl/actions/workflows/windows.yml)
[![Documentation Status](https://readthedocs.org/projects/openfl/badge/?version=latest)](https://openfl.readthedocs.io/en/latest/?badge=latest)
Expand All @@ -11,6 +11,7 @@
[![License](https://img.shields.io/badge/License-Apache%202.0-brightgreen.svg)](https://opensource.org/licenses/Apache-2.0)
[![Citation](https://img.shields.io/badge/cite-citation-brightgreen)](https://arxiv.org/abs/2105.06413)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/intel/openfl/blob/develop/openfl-tutorials/interactive_api/numpy_linear_regression/workspace/SingleNotebook.ipynb)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/6599/badge)](https://bestpractices.coreinfrastructure.org/projects/6599)

OpenFL is a Python 3 framework for Federated Learning. OpenFL is designed to be a _flexible_, _extensible_ and _easily learnable_ tool for data scientists. OpenFL is hosted by Intel, aims to be community-driven, and welcomes contributions back to the project.

Expand Down
Binary file added docs/images/CI_details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 14 additions & 2 deletions openfl-tutorials/Federated_FedProx_PyTorch_MNIST_Tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,13 @@
"outputs": [],
"source": [
"#Run experiment, return trained FederatedModel\n",
"final_fl_model = fx.run_experiment(collaborators,{'aggregator.settings.rounds_to_train':5})"
"final_fl_model = fx.run_experiment(\n",
" collaborators,\n",
" {\n",
" 'aggregator.settings.rounds_to_train': 5,\n",
" 'collaborator.settings.opt_treatment': 'CONTINUE_GLOBAL',\n",
" }\n",
")"
]
},
{
Expand Down Expand Up @@ -473,7 +479,13 @@
"outputs": [],
"source": [
"#Run experiment, return trained FederatedModel\n",
"final_fl_model = fx.run_experiment(collaborators,{'aggregator.settings.rounds_to_train':20})"
"final_fl_model = fx.run_experiment(\n",
" collaborators,\n",
" {\n",
" 'aggregator.settings.rounds_to_train': 20,\n",
" 'collaborator.settings.opt_treatment': 'CONTINUE_GLOBAL',\n",
" }\n",
")"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@
"!pip install -r requirements_workflow_interface.txt\n",
"\n",
"# Uncomment this if running in Google Colab\n",
"#pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#!pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#import os\n",
"#os.environ[\"USERNAME\"] = \"colab\""
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"!pip install -r requirements_workflow_interface.txt\n",
"\n",
"# Uncomment this if running in Google Colab\n",
"#pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#!pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#import os\n",
"#os.environ[\"USERNAME\"] = \"colab\""
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@
"!pip install -r requirements_workflow_interface.txt\n",
"\n",
"# Uncomment this if running in Google Colab\n",
"#pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#!pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#import os\n",
"#os.environ[\"USERNAME\"] = \"colab\""
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
"!pip install -r requirements_workflow_interface.txt\n",
"\n",
"# Uncomment this if running in Google Colab\n",
"#pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#!pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#import os\n",
"#os.environ[\"USERNAME\"] = \"colab\""
]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
"source": [
"# Workflow Interface 301: Watermarking\n",
"\n",
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/intel/openfl/blob/develop/openfl-tutorials/experimental/Workflow_Interface_301_MNIST_Watermaring.ipynb)"
"[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/intel/openfl/blob/develop/openfl-tutorials/experimental/Workflow_Interface_301_MNIST_Watermarking.ipynb)"
]
},
{
Expand Down Expand Up @@ -57,7 +57,7 @@
"\n",
"\n",
"# Uncomment this if running in Google Colab\n",
"#pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#!pip install -r https://raw.githubusercontent.com/intel/openfl/develop/openfl-tutorials/experimental/requirements_workflow_interface.txt\n",
"#import os\n",
"#os.environ[\"USERNAME\"] = \"colab\""
]
Expand Down
Loading