Understanding the Role of Momentum in Stochastic Gradient Methods

To reproduce experiments performed in the paper, see code and comments in experiments.ipynb.

To run logistic regression on MNIST or ResNet-18 on CIFAR, you can use scripts in lr_on_mnist and resnet_on_cifar correspondingly. Run python main.py --help for list of available hyperparameters in both scripts.

To be able to run the code, you need to install the following Python packages: numpy, matplotlib, pytorch (v0.4), tensorboardx