Skip to content

Latest commit

 

History

History
135 lines (101 loc) · 8.08 KB

README.md

File metadata and controls

135 lines (101 loc) · 8.08 KB
Raw

Towards Efficient 3D Object Detection with Knowledge Distillation

Jihan Yang, Shaoshuai Shi, Runyu Ding, Zhe Wang, Xiaojuan Qi

This repository contains the official implementation of Towards Efficient 3D Object Detection with Knowledge Distillation, NeurIPS 2022

Changelog

[2022-09-] Code release for Waymo results.

Introduction

Our code is based on OpenPCDet v0.5.2. More updates on OpenPCDet are supposed to be compatible with our code.

Model Zoo

Waymo Open Dataset

Basic Models without KD

Similar to OpenPCDet, all models in the following table are trained with a single frame of 20% data (~32k frames) of all the training samples on 8 GTX 1080Ti GPUs. Note that the validation are also carried on the 20% validation set with DATA_CONFIG.SAMPLED_INTERVAL.test 5 (similar performance with 100% validation set).

Model (20% data) LEVEL2 mAPH Flops (G) Acts (M) Latency (ms)
CP-Voxel 64.29 114.8 101.9 125.70
CP-Voxel-S 62.23 47.8 65.7 97.99
CP-Voxel-XS 61.14 36.9 58.4 88.19
CP-Voxel-XXS 56.26 12.0 33.1 70.44
CP-Pillar 59.09 333.9 303.0 157.90
CP-Pillar-v0.4 57.55 212.9 197.7 103.37
CP-Pillar-v0.48 56.27 149.4 142.3 81.87
CP-Pillar-v0.64 52.81 85.1 88.0 54.52

Sparse Distillation Models

Model (20% data) LEVEL2 mAPH Gains
CP-Voxel-S + SparseKD 64.25 +2.0
CP-Voxel-XS + SparseKD 63.53 +2.4
CP-Voxel-XXS + SparseKD 59.28 +3.0
CP-Pillar-v0.4 + SparseKD 59.24 +1.7
CP-Pillar-v0.48 + SparseKD 58.53 +2.3
CP-Pillar-v0.64 + SparseKD 55.82 +3.0

Here we also provide the performance of several models trained on the full training set and validate on the full validation set.

Model (100% data) LEVEL2 mAPH
CP-Voxel 65.58
CP-Pillar 61.56
PV-RCNN++ 69.46
CP-Voxel-S + SparseKD 65.75
CP-Voxel-XS + SparseKD 64.83
CP-Voxel-XXS + SparseKD 60.93
CP-Pillar-v0.4 + SparseKD 61.60
CP-Pillar-v0.48 + SparseKD 60.95
CP-Pillar-v0.64 + SparseKD 58.89

Cross stage distillation

Model (20% data) LEVEL2 mAPH Flops (G) Acts (M) Latency (ms)
PV-RCNN++ 67.80 123.5 179.7 435.9
CP-Voxel 64.29 114.8 101.9 125.7
CP-Voxel + SparseKd 65.27 114.8 101.9 125.7

We could not publicly provide the above pretrained models due to Waymo Dataset License Agreement. To access these pretrained models, please email us your name, institute, a screenshot of the Waymo dataset registration confirmation mail, and your intended usage. Please send a second email if we don't get back to you in two days. Please note that Waymo open dataset is under strict non-commercial license, so we are not allowed to share the model with you if it will use for any profit-oriented activities.

Latency on Different Hardware

latency

To facilitate further researches, we provide latency measured in milliseconds as follows:

Model 1060 + Spconv2.x 1080Ti + Spconv2.x A100 + Spconv2.x
SECOND 84.56 50.65 40.44
PointPillar 129.12 68.61 44.14
CP-Voxel 125.70 74.25 56.13
CP-Voxel-S 97.99 65.68 39.40
CP-Voxel-XS 88.19 61.41 36.65
CP-Voxel-XXS 70.44 53.86 38.01
CP-Pillar 157.90 77.29 51.94
CP-Pillar-v0.4 103.37 56.32 57.07
CP-Pillar-v0.48 81.87 49.64 27.44
CP-Pillar-v0.64 54.52 36.23 19.56

Installation

Please refer to INSTALL.md for the installation of OpenPCDet.

Getting Started

Please refer to GETTING_STARTED.md to learn more usage about this project.

License

Our code is released under the Apache 2.0 license.

Acknowledgement

Our code is heavily based on OpenPCDet. Thanks OpenPCDet Development Team for their awesome codebase.

Citation

If you find this project useful in your research, please consider cite:

@inproceedings{yang2022towards,
    title={Towards Efficient 3D Object Detection with Knowledge Distillation},
    author={Yang, Jihan and Shi, Shaoshuai and Ding, Runyu and Wang, Zhe and Qi, Xiaojuan},
    booktitle={Advances in Neural Information Processing Systems},
    year={2022}
}

@misc{openpcdet2020,
    title={OpenPCDet: An Open-source Toolbox for 3D Object Detection from Point Clouds},
    author={OpenPCDet Development Team},
    howpublished = {\url{https://github.com/open-mmlab/OpenPCDet}},
    year={2020}
}