General purpose, dual githup repository for evaluating Lipschitz constants of ReLU networks. This project originally developed out of a mixed-integer programming approach for exactly computing Lipschitz constants of neural networks with ReLU nonlinearities. Much of the same machinery was repurposed for an approximate Lipschitz approximation for neural networks with a larger class of nonlinearities and larger output dimension.
Ultimately, this repo handles the code for the following two papers:
- Exactly Computing the Local Lipschitz Constant of ReLU Networks. NeurIPS'20
- Provable Lipschitz Certification for Generative Models. ICML'21
- 10/24/2022: (way late) updated tutorials to include ZLip support.
- 07/09/2021: ArXiv Release for ZLip Paper.
- 05/11/2021: ZLip Paper accepted at ICML 2021.
- 09/26/2020: LipMIP Paper accepted at NeurIPS 2020.
- 03/04/2020: ArXiv Release and Version 0.1 deployed
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes.
Requisite python packages are contained within the file requirements.txt. We use pytorch for automatic differentiation and handy neural network utilities. We also use Gurobi to solve mixed-integer programs.
We now provide detailed installation instructions: 1) Clone this repository and all submodules.
$ git clone https://github.com/revbucket/lipMIP.git
$ git submodule update --init --recursive
2) Install pip dependencies. We recommend that dependencies be installed within a virtual environment. Instructions to set this up are here. With the virtual environment activated, we start by installing all pip-based python packages:
$ cd lipMIP
$ pip3 install -r requirements.txt
3) Install Gurobi and Gurobipy. First download the most recent version of the Gurobi optimizer (accessible with a free academic license). Gurobi Website.
Then add the following environment variables to a file that runs on terminal startup (e.g. ~/.bash_profile):
export GUROBI_HOME="/your/path/to/gurobi/linux64"
export PATH="${PATH}:${GUROBI_HOME}/bin"
export LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:${GUROBI_HOME}/lib"
And install the license obtained from the Gurobi website with the command:
$ grbgetkey aaaa0000-0000-0000-0000-000000000000
Finally install and test the gurobipy package to your python virtual environment by calling
$ cd $GUROBI_HOME
$ python3 setup.py install
$ python3 -c "import gurobipy"
4) Install Mosek and Matlab. To evaluate against LipSDP, we'll need to install Matlab, the python-link to Matlab, and the Mosek solver. Follow the requirements section of LipSDP for more details.
5) (optional) Add kernel representing this virtual environment to Jupyter. We use jupyter notebook quite frequently for sandboxing purposes. To reflect the virtual environment you've built within jupyter, run the following command:
$ python3 -m ipykernel install --name lipmip
We have included three tutorials for getting started with this repository. The first one describes basic functionality of neural nets, datasets, and training. The second one describes basic and more advanced usage of LipMIP. The third one describes how to compare LipMIP to other leading Lipschitz estimation techniques (including my other method, ZLip).
With the codebase installed, run the ipython notebooks provided in tutorials/ directory.
$ cd lipMIP
$ jupyter notebook and navigate to the tutorials/ directory and interact with each tutorial file.
This project is licensed under the MIT License - see the LICENSE.md file for details