mnp: Mininet Packaging Tools Docs

Introduction

mnp, available here, is a command-line tool developed during Google Summer of Code 2013 with these purposes:

• Help Mininet users download a Mininet modules from a sharing website, including the dependencies with as few commands as possible.
• Help Mininet developers package, upload, and share their Mininet modules on a sharing website.
• Make it easy to do some basic querying to the sharing website from the command line.

The sharing website, a modified PyPI clone with non-standard behaviour, is developed with djangopypi2, an existing PyPI clone, as a base during Google Summer of Code 2013 as well.

Mininet Packaging Format

Mininet packaging format is just a plain old Python packaging format based on setuptools and distutils.

mnp Commands

• download: Download specified packages
• upload: Package and upload the package from the current directory
• list: List all available packages
• search: Search for packages
• docs: Retrieve the documentation of a package
• info: Print the essential information (author, version, summary etc.) of a package

Other Features

• GitHub authentication support for upload command

Implementation

The commands supported by mnp are implemented internally by reusing existing tools as much as possible.

Commands

download

Implemented by calling pip from the command line.

upload

Implemented by calling setup.py from the command line which in turn calls setuptools.

list, search, docs, info

Implemented by making an XML-RPC request to the sharing website.

Others

GitHub authentication support for upload command

Implementing the support for GitHub authentication is the most technically challenging part of the development of mnp.

After trying out several approaches, I arrived at the conclusion that the most elegant way to implement GitHub authentication is by implementing additional commands to setup.py by creating additional cmdclass.

By reading through the source code of both setuptools and distutils for both register and upload commands, I managed to find the proper methods to override in order to read in users' specified GitHub username from .pypirc. If the GitHub password is not specified in .pypirc, then the user will be asked for it when the command is executed. Once GitHub username and password is obtained, the authentication mechanism will be used to authenticate to the sharing website.

Implementing it this way, however, means that users who wanted to use GitHub account to submit their packages must always import and specify the cmdclass-es in their setup.py files. This is very inconvenient as the setup.py files are quite complex enough as it is, so I decided to monkey-patch the setup.py to patch the setup() function to automatically include the cmdclass-es written to support GitHub authentication.

This monkey-patch mechanism can then be easily activated simply by writing import mnp.patch at the top of the setup.py files. The written packages must also depend on mnp in order to be able to import the patch in the first place, but the syntax to achieve this is much friendlier than the syntax to declare a new cmdclass.

GitHub Account Authentication Mechanism

Given the GitHub username and password, mnp will send request to GitHub users API to convert the username and password combination to a user id.

Given the GitHub username and password, mnp will also get the list of authorization tokens associated with the account via the GitHub OAuth API. From the list of authorization tokens, mnp will find the one token associated with the sharing website by finding the one having the client_id property set to the application id of the sharing website.

Once both the user id and the authorization token are retrieved, the user id is used as the username while the authorization token is hashed and then used as the password to authenticate to the sharing website.

Unit Testing

References

PyPI XML-RPC