Skip to content

Latest commit

 

History

History
90 lines (70 loc) · 2.86 KB

README.md

File metadata and controls

90 lines (70 loc) · 2.86 KB
Raw

3D Multibodies

Code for the paper 3D Multibodies, NeurIPS 2020 Spotlight.

Installation

cd 3D-Multibodies
mkdir data

// Clone the SPIN data
wget http://visiondata.cis.upenn.edu/spin/data.tar.gz && tar -xvf data.tar.gz && rm data.tar.gz
wget http://visiondata.cis.upenn.edu/spin/dataset_extras.tar.gz && tar -xvf dataset_extras.tar.gz --directory data

// Clone the SPIN pretrained checkpoint
mkdir -p data/pretrained
wget http://visiondata.cis.upenn.edu/spin/model_checkpoint.pt --directory-prefix=data/pretrained

Download the 3DMB data from Google Drive

  • Add files from 3dmb to a directory data/3dmb
  • Add files from crops to a directory data/crops
  • Add files from pretrained to the data/pretrained used above

Download SMPL models as in src/config.py and place them in data/smpl

Evaluation

cd src
CUDA_VISIBLE_DEVICES=0 python experiment.py \ 
    --visualize_interval 100 \
    --test_batch_size 75 \
    --num_workers 12 \
    --evaluate True \
    --MODEL.init_flow ../data/pretrained/normflow/model_epoch_00000412.pth \
    --DATASET.dataset_name h36m_only \
    --DATASET.ambiguous False \
    --exp_dir ../data/pretrained/standard

For H36M or AH36M set

--DATASET.dataset_name h36m_only

For 3DPW or A3DPW set

--DATASET.dataset_name 3dpw_only

For AH36M or A3DPW experiments set

--DATASET.ambiguous True \
--exp_dir ../data/pretrained/ambiguous

Run All

To run the complete set of evaluations, install tmux and use the script

cd src
bash scripts/run_eval_ec2.bash

Expected Results

01_mpjpe 01_reco 05_mpjpe 05_reco 10_mpjpe 10_reco 25_mpjpe 25_reco
h36m 61.517 41.6263 59.7716 41.9884 59.1608 42.1022 58.236 42.1811
3dpw_WEIGHT 93.7943 59.8731 81.7215 56.714 78.7641 56.2927 75.2801 55.2845
ah36m_WEIGHT 103.612 67.8314 95.3061 65.4571 92.4205 64.5733 88.7481 63.1861
a3dpw_WEIGHT 149.671 78.3031 125.612 74.2592 116.774 73.5848 107.465 71.9949

Citation

@inproceedings{biggs2020multibodies,
  author = "Biggs, Benjamin and Ehrhart, S{\'{e}}bastien and Joo, Hanbyul and Graham, Benjamin and Vedaldi, Andrea and Novotny, David",
  title = "{3D} Multibodies: Fitting Sets of Plausible {3D} Models to Ambiguous Image Data",
  booktitle = "NeurIPS",
  year = "2020",
}

Acknowledgements

Much of the code here has been borrowed from the excellent SPIN repository.