Qi Fan, Wenjie Pei, Yu-Wing Tai, Chi-Keung Tang
The codebase contains the official code of our paper Self-Support Few-Shot Semantic Segmentation, ECCV 2022.
中文解读: ECCV 2022 | SSP: 自支持匹配的小样本任务新**
Pretrained model: ResNet-50 | ResNet-101
Dataset: Pascal images and ids | Semantic segmentation annotations
You can follow HSNet to prepare the MS COCO and FSS-1000 datasets.
../ # parent directory
├── ./SSP # current (project) directory
| ├── codes # various codes
| └── ./pretrained # pretrained model directory
| ├── resnet50.pth
| └── resnet101.pth
└── Datasets_SSP/
├── VOC2012/ # PASCAL VOC2012 devkit
│ ├── Annotations/
│ ├── ImageSets/
│ ├── ...
│ └── SegmentationClassAug/
├── COCO2014/
│ ├── annotations/
│ │ ├── train2014/ # (dir.) training masks (from Google Drive)
│ │ ├── val2014/ # (dir.) validation masks (from Google Drive)
│ │ └── ..some json files..
│ ├── train2014/
│ └── val2014/
└── FSS-1000/ # (dir.) contains 1000 object classes
├── abacus/
├── ...
└── zucchini/
You can adapt the scripts of all.sh, test.sh and hsnet_test.sh (for the HSNet evaluation protocol) to train and evaluate your models.
CUDA_VISIBLE_DEVICES=0,1 python -W ignore main.py \
--dataset pascal --data-root [Your Pascal Path] \
--backbone resnet50 --fold 0 --shot 1
You may change the backbone from resnet50 to resnet101, change the fold from 0 to 1/2/3, or change the shot from 1 to 5 for other settings.
You can add/remove --refine to enable/disable the self-support refinement.
| Method | Setting | Backbone | SSP Refine | Fold 0 | Fold 1 | Fold 2 | Fold 3 | Mean |
|---|---|---|---|---|---|---|---|---|
| Baseline | 1-shot | ResNet-50 | No | 54.9 | 66.5 | 61.7 | 48.3 | 57.9 |
| Baseline | 1-shot | ResNet-101 | No | 57.2 | 68.5 | 61.3 | 53.3 | 60.1 |
| Baseline | 5-shot | ResNet-50 | No | 61.6 | 70.3 | 70.5 | 56.4 | 64.7 |
| Baseline | 5-shot | ResNet-101 | No | 64.2 | 74.0 | 71.5 | 61.3 | 67.8 |
| SSP (Ours) | 1-shot | ResNet-50 | Yes | 61.4 | 67.8 | 66.5 | 50.9 | 61.7 |
| SSP (Ours) | 1-shot | ResNet-101 | Yes | 63.2 | 70.4 | 68.5 | 56.3 | 64.6 |
| SSP (Ours) | 5-shot | ResNet-50 | Yes | 67.5 | 72.3 | 75.2 | 62.1 | 69.3 |
| SSP (Ours) | 5-shot | ResNet-101 | Yes | 70.9 | 77.1 | 78.9 | 66.1 | 73.3 |
| Method | Setting | Backbone | Eval Protocol | Fold 0 | Fold 1 | Fold 2 | Fold 3 | Mean |
|---|---|---|---|---|---|---|---|---|
| SSP (Ours) | 1-shot | ResNet-50 | Ours | 46.4 | 35.2 | 27.3 | 25.4 | 33.6 |
| SSP (Ours) | 1-shot | ResNet-101 | Ours | 50.4 | 39.9 | 30.6 | 30.0 | 37.7 |
| SSP (Ours) | 5-shot | ResNet-50 | Ours | 53.9 | 42.0 | 36.0 | 33.7 | 41.4 |
| SSP (Ours) | 5-shot | ResNet-101 | Ours | 57.8 | 47.0 | 40.2 | 39.9 | 46.2 |
| SSP (Ours) | 1-shot | ResNet-50 | HSNet | 35.5 | 39.6 | 37.9 | 36.7 | 37.4 |
| SSP (Ours) | 1-shot | ResNet-101 | HSNet | 39.1 | 45.1 | 42.7 | 41.2 | 42.0 |
| SSP (Ours) | 5-shot | ResNet-50 | HSNet | 40.6 | 47.0 | 45.1 | 43.9 | 44.1 |
| SSP (Ours) | 5-shot | ResNet-101 | HSNet | 47.4 | 54.5 | 50.4 | 49.6 | 50.2 |
This codebase is built based on MLC's baseline code and we borrow HSNet's evaluation protocol for the MS COCO dataset. We thank MLC and other FSS works for their great contributions.
Few-shot image/video object detection: FewX
@inproceedings{fan2021fsvod,
title={Few-Shot Video Object Detection},
author={Fan, Qi and Tang, Chi-Keung and Tai, Yu-Wing},
booktitle={ECCV},
year={2022}
}
@inproceedings{fan2020cpmask,
title={Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation},
author={Fan, Qi and Ke, Lei and Pei, Wenjie and Tang, Chi-Keung and Tai, Yu-Wing},
booktitle={ECCV},
year={2020}
}
@inproceedings{fan2020fsod,
title={Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector},
author={Fan, Qi and Zhuo, Wei and Tang, Chi-Keung and Tai, Yu-Wing},
booktitle={CVPR},
year={2020}
}@inproceedings{fan2022ssp,
title={Self-Support Few-Shot Semantic Segmentation},
author={Fan, Qi and Pei, Wenjie and Tai, Yu-Wing and Tang, Chi-Keung},
journal={ECCV},
year={2022}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent