Skip to content

Latest commit

 

History

History
44 lines (30 loc) · 2.15 KB

README.md

File metadata and controls

44 lines (30 loc) · 2.15 KB
Raw

GULF: GUided Learning through successive Functional gradient optimization

This repository provides the author implementation of GULF, described in [Johnson & Zhang, ICML2020]. It also provides the author implementation (in pyTorch) of deep pyramid convolutional neural networks (DPCNN) [Johnson & Zhang ACL2017] used in the GULF experiments on text classification.

Requirements

  • Python version 3
  • pyTorch 1.2.0 and torchvision 0.4.0 (or higher)
  • pip install -r requirements.txt

Examples

  • To perform GULF2 with ini:random with ResNet-28 on CIFAR100: python3 train_cifar.py
  • To change the initialization to 'ini:base': python3 train_cifar.py --ini_type iniBase
  • To write test error etc. to a file in the csv format: python3 train_cifar.py --csv_fn results.csv
  • To get help: python3 train_cifar.py -h

Similarly, train_svhn.py, train_imgnet.py, and train_yelp can be used.
train_yelp.py for sentiment classification and DPCNN are explained here.

Example configurations

code CPU cores CPU memory GPU
train_cifar.py 3 20GB 1
train_svhn.py 2 20GB 1
train_imgnet.py, resnet50 8 128GB 2
train_imgnet.py, wrn50-2 12 128GB 4
train_yelp.py 1 24GB 1

GPU device memory: 12GB

Notes

  • The code uses a GPU whenever it is available. To avoid use of GPUs even when it is available, empty CUDA_VISIBLE_DEVICES via shell before calling python.

      export CUDA_VISIBLE_DEVICES=""

  • The code writes a lot to stdout, and so it is recommended to redirect it to a file.

References

[Johnson & Zhang, 2020] Guided Learning of Nonconvex Models through Successive Functional Gradient Optimization. Rie Johnson and Tong Zhang. ICML-2020.
[Johnson & Zhang, 2017] Deep pyramid convolutional neural networks for text categorization. Rie Johnson and Tong Zhang. ACL-2017.