Polygraphy is a toolkit designed to assist in running and debugging deep learning models in various frameworks. It includes a Python API and a command-line interface (CLI) built using this API.
Among other things, Polygraphy lets you:
- Run inference among multiple backends, like TensorRT and ONNX-Runtime, and compare results (example: API, CLI)
- Convert models to various formats, e.g. TensorRT engines with post-training quantization (example: API, CLI)
- View information about various types of models (example: CLI)
- Modify ONNX models on the command-line:
- Isolate faulty tactics in TensorRT (example: CLI)
IMPORTANT: Polygraphy does not support Python 2.X. All the instructions here assume you are using Python 3 or later.
python -m pip install colored polygraphy --extra-index-url https://pypi.ngc.nvidia.com
NOTE: When using this method, the command-line toolkit will be installed into ${HOME}/.local/bin
by default.
Make sure to add this directory to your PATH
environment variable.
make install
Make sure you are allowed to execute scripts on your system then run:
.\install.ps1
- Install prerequisites:
python -m pip install wheel
- Build a wheel:
python setup.py bdist_wheel
-
Install the wheel manually from outside the repository:
On Linux, run:
python -m pip install Polygraphy/dist/polygraphy-*-py2.py3-none-any.whl
On Windows, using Powershell, run:
$wheel_path = gci -Name Polygraphy\dist python -m pip install Polygraphy\dist\$wheel_path
NOTE: It is strongly recommended to install the
colored
module for colored output from Polygraphy, as this can greatly improve readability:python -m pip install colored
Polygraphy has no hard-dependencies on other Python packages. However, much of the functionality included does require other Python packages.
It's non-trivial to determine all the packages that will be required ahead of time, since it depends on exactly what functionality is being used.
To make this easier, Polygraphy can optionally automatically install or upgrade dependencies at runtime, as they are needed.
To enable this behavior, set the POLYGRAPHY_AUTOINSTALL_DEPS
environment variable to 1
or
polygraphy.config.AUTOINSTALL_DEPS = True
using the Python API.
NOTE: By default, dependencies will be installed using the current interpreter, and may overwrite existing
packages. The default installation command, which is python -m pip install
, can be overriden by setting
the POLYGRAPHY_INSTALL_CMD
environment variable, or setting polygraphy.config.INSTALL_CMD
using the Python API.
Each backend
directory includes a requirements.txt
file that specifies the minimum set of packages
it depends on. This does not necessarily include all packages required for all the functionality provided
by the backend, but does serve as a good starting point.
You can install the requirements for whichever backends you're interested in with:
python -m pip install -r polygraphy/backend/<name>/requirements.txt
If additional packages are required, warnings or errors will be logged. You can install the additional packages manually with:
python -m pip install <package_name>
For details on the various tools included in the Polygraphy toolkit, see the CLI User Guide.
For more information on the Polygraphy Python API, including a high-level overview and the Python API reference documentation, see the API directory.
For examples of both the CLI and Python API, see the examples directory.
For how-to guides, see the how-to guides directory.
For information on how you can contribute to this project, see CONTRIBUTING.md