(ICCV 2021) Multitask AET with Orthogonal Tangent Regularity for Dark Object Detection (paper) (supp)
When Human Vision Meets Machine Vision (compare with enhancement methods):
Physics-based low-light degrading transformation (unprocess -- degradation -- ISP):
python 3.7
pytorch 1.6.0
mmcv 1.1.5
matplotlib opencv-python Pillow tqdm
| dataset | model | size | logs |
|---|---|---|---|
| MAET-COCO (ours) 80 class | (google drive) (baiduyun, passwd:1234) | 489.10 MB | - |
| MAET-EXDark (ours) (77.7) 20 class | (google drive) (baiduyun, passwd:1234) | 470.26 MB | google drive |
| EXDark (76.8) 20 class | ([google drive]) (baiduyun, passwd:1234) | 470.26 MB | - |
| EXDark (MBLLEN) (76.3) 20 class | (google drive) (baiduyun, passwd:1234) | 470.26 MB | - |
| EXDark (Kind) (76.3) 20 class | (google drive) (baiduyun, passwd:1234) | 470.26 MB | - |
| EXDark (Zero-DCE) (76.9) 20 class | (google drive) (baiduyun, passwd:1234) | 470.26 MB | - |
| MAET-UG2-DarkFace (ours) (56.2) 1 class | (google drive) (baiduyun, passwd:1234) | 469.81 MB | - |
Step-1:
For MS COCO Dataset: Download COCO 2017 dataset.
For EXDark Dataset: Download EXDark (include EXDark enhancement by MBLLEN, Zero-DCE, KIND) in VOC format from google drive or baiduyun, passwd:1234. The EXDark dataset should be look like:
EXDark
│
│
└───JPEGImages
│ │───IMGS (original low light)
│ │───IMGS_Kind (imgs enhancement by [Kind, mm2019])
│ │───IMGS_ZeroDCE (imgs enhancement by [ZeroDCE, cvpr 2020])
│ │───IMGS_MEBBLN (imgs enhancement by [MEBBLN, bmvc 2018])
│───Annotations
│───main
│───label
For UG2-DarkFace Dataset: Download UG2 in VOC format from google drive or baiduyun, passwd:1234. The UG2-DarkFace dataset should be look like:
UG2
│
└───main
│───xml
│───label
│───imgs
Step-2: Cd in "your_project_path", and do set-up process (see mmdetection if you want find details):
git clone [email protected]:cuiziteng/ICCV_MAET.git
cd "your project page"
pip install -r requirements/build.txt
pip install -v -e . # or "python setup.py develop"
Step-3: Change the data place line1 and line2 to your own COCO and EXDark path, and line3 to your own UG2-DarkFace path.
Testing MAET-YOLOV3 on (low-light) COCO dataset
python tools/test.py configs/MAET_yolo/maet_yolo_coco_ort.py [COCO model path] --eval bbox --show-dir [save dir]
Testing MAET-YOLOV3 on EXDark dataset
python tools/test.py configs/MAET_yolo/maet_yolo_exdark.py [EXDark model path] --eval mAP --show-dir [save dir]
Testing MAET-YOLOV3 on UG2-DarkFace dataset
python tools/test.py configs/MAET_yolo/maet_yolo_ug2.py [UG2-DarkFace model path] --eval mAP --show-dir [save dir]
Comparative Experiment
Testing YOLOV3 on EXDark dataset enhancement by MEBBLN/ Kind/ Zero-DCE
python tools/test.py configs/MAET_yolo/yolo_mbllen.py (yolo_kind.py, yolo_zero_dce.py) [MEBBLN/ Kind/ Zero-DCE model] --eval mAP --show-dir [save dir]
Setp-1: Pre-train MAET-COCO model (273 epochs on 4 GPUs): (if use other GPU number, please reset learining rate)
CUDA_VISIBLE_DEVICES=0,1,2,3 PORT=[port number] bash ./tools/dist_train_maet.sh configs/MAET_yolo/maet_yolo_coco_ort.py 4
Setp-2 (EXDark): Fine-tune on EXDark datastet (25epoch on 1 GPU):
python tools/train.py configs/MAET_yolo/maet_yolo_exdark.py --gpu-ids [gpu id] --load-from [COCO model path]
Setp-2 (UG2-DarkFace): Fine-tune on UG2-DarkFace datastet (20epoch on 1 GPU):
python tools/train.py configs/MAET_yolo/maet_yolo_ug2.py --gpu-ids [gpu id] --load-from [COCO model path]
Comparative Experiment
Fine-tune EXDark dataset enhancement by MEBBLN/ Kind/ Zero-DCE (25epoch on 1 GPU) on well-trained normal COCO model (608x608) for fairness
python tools/train.py configs/MAET_yolo/yolo_mbllen.py (yolo_kind.py, yolo_zero_dce.py) --gpu-ids [gpu id]
| class | gts | dets | recall | ap |
|---|---|---|---|---|
| Bicycle | 212 | 773 | 0.920 | 0.831 |
| Boat | 289 | 942 | 0.900 | 0.785 |
| Bottle | 282 | 1217 | 0.879 | 0.756 |
| Bus | 135 | 331 | 0.970 | 0.929 |
| Car | 597 | 1788 | 0.915 | 0.831 |
| Cat | 183 | 579 | 0.885 | 0.734 |
| Chair | 466 | 2132 | 0.854 | 0.713 |
| Cup | 366 | 1086 | 0.880 | 0.790 |
| Dog | 207 | 631 | 0.918 | 0.798 |
| Motorbike | 233 | 946 | 0.884 | 0.772 |
| People | 1562 | 4353 | 0.906 | 0.811 |
| Table | 333 | 1880 | 0.805 | 0.570 |
| mAP | 0.777 |
If our work help to your research, please cite our paper~ ^-^, thx.
@InProceedings{Cui_2021_ICCV,
author = {Cui, Ziteng and Qi, Guo-Jun and Gu, Lin and You, Shaodi and Zhang, Zenghui and Harada, Tatsuya},
title = {Multitask AET With Orthogonal Tangent Regularity for Dark Object Detection},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {2553-2562}
}
The code is largely borrow from mmdetection and unprocess, Thx to their wonderful works~
MMdetection: mmdetection (v2.7.0)
Unprocessing Images for Learned Raw Denoising: unprocess
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent