Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation

This is an official implementation of the paper "Decomposed Knowledge Distillation for Class-Incremental Semantic Segmentation", accepted to NeurIPS 2022.

For more information, please checkout the project site [website] and our paper [arXiv / OpenReview].

Pre-requisites

This repository has been tested with the following libraries:

• Python (3.6)
• Pytorch (1.8.1)

Getting Started

Datasets

PASCAL VOC 2012

We use augmented 10,582 training samples and 1,449 validation samples for PASCAL VOC 2012. You can download the original dataset in here. To train our model with augmented samples, please download labels of augmented samples ('SegmentationClassAug') and file names ('train_aug.txt'). The structure of data path should be organized as follows:

└── /dataset/VOC2012
    ├── Annotations
    ├── ImageSets
    │   └── Segmentation
    │       ├── train_aug.txt
    │       └── val.txt
    ├── JPEGImages
    ├── SegmentationClass
    └── SegmentationClassAug

ADE20K

We use 20,210 training samples and 2,000 validation samples for ADE20K. You can download the dataset in here. The structure of data path should be organized as follows:

└── /dataset/ADEChallengeData2016
    ├── annotations
    ├── images
    ├── objectInfo150.txt
    └── sceneCategories.txt

Training

PASCAL VOC 2012

# An example srcipt for 15-5 overlapped setting of PASCAL VOC

GPU=0,1,2,3
BS=8  # Total 32
SAVEDIR='saved_voc'

TASKSETTING='overlap'  # or 'disjoint'
TASKNAME='15-5' # or ['15-1', '19-1', '10-1', '5-3']
EPOCH=60
INIT_LR=0.001
LR=0.0001
INIT_POSWEIGHT=2
MEMORY_SIZE=0  # 100 for DKD-M

NAME='DKD'
python train_voc.py -c configs/config_voc.json \
-d ${GPU} --multiprocessing_distributed --save_dir ${SAVEDIR} --name ${NAME} \
--task_name ${TASKNAME} --task_setting ${TASKSETTING} --task_step 0 --lr ${INIT_LR} --bs ${BS} --pos_weight ${INIT_POSWEIGHT}

python train_voc.py -c configs/config_voc.json \
-d ${GPU} --multiprocessing_distributed --save_dir ${SAVEDIR} --name ${NAME} \
--task_name ${TASKNAME} --task_setting ${TASKSETTING} --task_step 1 --lr ${LR} --bs ${BS} --freeze_bn --mem_size ${MEMORY_SIZE}

ADE20K

# An example srcipt for 50-50 overlapped setting of ADE20K

GPU=0,1,2,3
BS=6  # Total 24
SAVEDIR='saved_ade'

TASKSETTING='overlap'
TASKNAME='50-50' # or ['100-10', '100-50']
EPOCH=100
INIT_LR=0.0025
LR=0.00025
MEMORY_SIZE=0 # 300 for DKD-M

NAME='DKD'
python train_ade.py -c configs/config_ade.json \
-d ${GPU} --multiprocessing_distributed --save_dir ${SAVEDIR} --name ${NAME} \
--task_name ${TASKNAME} --task_setting ${TASKSETTING} --task_step 0 --lr ${INIT_LR} --bs ${BS}

python train_ade.py -c configs/config_ade.json \
-d ${GPU} --multiprocessing_distributed --save_dir ${SAVEDIR} --name ${NAME} \
--task_name ${TASKNAME} --task_setting ${TASKSETTING} --task_step 1 --lr ${LR} --bs ${BS} --freeze_bn --mem_size ${MEMORY_SIZE}

python train_ade.py -c configs/config_ade.json \
-d ${GPU} --multiprocessing_distributed --save_dir ${SAVEDIR} --name ${NAME} \
--task_name ${TASKNAME} --task_setting ${TASKSETTING} --task_step 2 --lr ${LR} --bs ${BS} --freeze_bn --mem_size ${MEMORY_SIZE}

Testing

PASCAL VOC 2012

python eval_voc.py -d 0 -r path/to/weight.pth

We provide pretrained weights and configuration files. The results should be:

Method (Overlapped)	VOC 19-1 (2 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 15-5 (2 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 15-1 (6 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 10-1 (11 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 5-3 (6 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$
DKD	56.95 / 76.13	67.17 / 73.95	57.46 / 70.50	57.21 / 60.43	61.32 / 58.98
DKD-M	68.14 / 77.04	68.96 / 74.84	64.09 / 72.95	64.89 / 66.20	65.02 / 63.32

Method (Disjoint)	VOC 19-1 (2 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 15-5 (2 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	VOC 15-1 (6 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$
DKD	57.98 / 75.90	64.35 / 72.21	54.21 / 68.48
DKD-M	67.12 / 76.84	65.04 / 72.53	60.32 / 70.78

ADE20K

python eval_ade.py -d 0 -r path/to/weight.pth

We provide pretrained weights and configuration files. The results should be:

Method	ADE 100-50 (2 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	ADE 100-10 (6 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$	ADE 50-50 (3 steps) $\text{hIoU}$ / $\text{mIoU}_{\text{all}}$
DKD	29.42 / 35.55	26.79 / 34.53	34.77 / 34.39
DKD-M	29.45 / 35.58	27.44 / 34.83	34.78 / 34.39

Acknowledgements

• This template is borrowed from pytorch-template.