-
Notifications
You must be signed in to change notification settings - Fork 0
mnp: Mininet Packaging Tools Docs
mnp, available here, is a command-line tool developed during Google Summer of Code 2013 with these purposes:
- Help Mininet users download a Mininet modules from a sharing website, including the dependencies with as few commands as possible.
- Help Mininet developers package, upload, and share their Mininet modules on a sharing website.
- Make it easy to do some basic querying to the sharing website from the command line.
The sharing website, a modified PyPI clone with non-standard behaviour, is developed with djangopypi2, an existing PyPI clone, as a base during Google Summer of Code 2013 as well.
Mininet packaging format is just a plain old Python packaging format based on setuptools and distutils.
-
download: Download specified packages -
upload: Package and upload the package from the current directory -
list: List all available packages -
search: Search for packages -
docs: Retrieve the documentation of a package -
info: Print the essential information (author, version, summary etc.) of a package
- GitHub authentication support for
uploadcommand
The commands supported by mnp are implemented internally by reusing existing tools as much as possible.
Implemented by calling pip from the command line.
Original command to download a package from a custom PyPI located on http://localhost:8000/simple/ with fallback on original PyPI:
pip install -i http://localhost:8000/simple/ --extra-index-url https://pypi.python.org/simple/ SomePackage
Implemented by calling setup.py from the command line which in turn calls setuptools.
Original command to upload a package to a mirror called mininet: python setup.py register -r mininet sdist upload -r mininet
Implemented by making an XML-RPC request to the sharing website.
Implementing the support for GitHub authentication is the most technically challenging part of the development of mnp.
After trying out several approaches, I arrived at the conclusion that the most elegant way to implement GitHub authentication is by implementing additional commands to setup.py by creating additional cmdclass.
By reading through the source code of both setuptools and distutils for both register and upload commands, I managed to find the proper methods to override in order to read in users' specified GitHub username from .pypirc. If the GitHub password is not specified in .pypirc, then the user will be asked for it when the command is executed. Once GitHub username and password is obtained, the authentication mechanism will be used to authenticate to the sharing website.
Implementing it this way, however, means that users who wanted to use GitHub account to submit their packages must always import and specify the cmdclass-es in their setup.py files. This is very inconvenient as the setup.py files are quite complex enough as it is, so I decided to monkey-patch the setup.py to patch the setup() function to automatically include the cmdclass-es written to support GitHub authentication.
This monkey-patch mechanism can then be easily activated simply by writing import mnp.patch at the top of the setup.py files. The written packages must also depend on mnp in order to be able to import the patch in the first place, but the syntax to achieve this is much friendlier than the syntax to declare a new cmdclass.
Given the GitHub username and password, mnp will send request to GitHub users API to convert the username and password combination to a user id.
Given the GitHub username and password, mnp will also get the list of authorization tokens associated with the account via the GitHub OAuth API. From the list of authorization tokens, mnp will find the one token associated with the sharing website by finding the one with the client_id property equal to the application id of the sharing website.
Once both the user id and the authorization token are retrieved, the user id is used as the username while the authorization token is hashed and then used as the password to authenticate to the sharing website via command line.
Note that for all this to work, the user must have use GitHub login at least once on the sharing website (i.e. not via command-line) so it has a chance to store the authorization token of the user. I think this is a reasonable requirement in order to use this feature.
The sharing website also needs to be modified in order to support both local account (i.e.: account registered on the sharing website itself) and GitHub authentication for upload command. I modified it to initially use the username and password to authenticate as local account first. If it fails to authenticate as a local account, then it checks if it is a GitHub account.