shenyunhang / na-fwebsod Goto Github PK

Noise-Aware Fully Webly Supervised Object Detection

Home Page: https://github.com/shenyunhang/NA-fWebSOD

License: Apache License 2.0

CMake 2.25% Makefile 0.03% Shell 0.31% Python 78.20% MATLAB 0.12% C++ 8.57% Cuda 10.48% Dockerfile 0.05%

webly-supervised-detection object-detection webly-supervised-learning weakly-supervised-learning weakly-supervised-object-detection

na-fwebsod's Introduction

Noise-Aware Fully Webly Supervised Object Detection

By Yunhang Shen, Rongrong Ji, Zhiwei Chen, Xiaopeng Hong, Feng Zheng, Jianzhuang Liu, Mingliang Xu, Qi Tian.

CVPR 2020 Paper.

This project is based on Detectron.

Introduction

License

NA-fWebSOD is released under the Apache 2.0 license. See the NOTICE file for additional details.

Citing NA-fWebSOD

If you find NA-fWebSOD useful in your research, please consider citing:

@inproceedings{NA-fWebSOD_2020_CVPR,
	author = {Shen, Yunhang and Ji, Rongrong and Chen, Zhiwei and Hong, Xiaopeng and Zheng, Feng and Liu, Jianzhuang and Xu, Mingliang and Tian, Qi},
	title = {Noise-Aware Fully Webly Supervised Object Detection},
	booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
	year = {2020},
}

Installation

Requirements:

NVIDIA GPU, Linux, Python3.6
Caffe2 in pytorch v1.3.0, various standard Python packages, and the COCO API; Instructions for installing these dependencies are found below

Caffe2

Clone the pytorch repository:

# pytorch=/path/to/clone/pytorch
git clone https://github.com/pytorch/pytorch.git $pytorch
cd $pytorch
git checkout v1.3.0
git submodule update --init --recursive

Install Python dependencies:

pip3 install -r $pytorch/requirements.txt

Build caffe2:

cd $pytorch
sudo USE_OPENCV=On USE_LMDB=On BUILD_BINARY=On python3 setup.py install

Other Dependencies

Install the COCO API:

pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

Install the pycococreator:

pip3 install git+git://github.com/waspinator/[email protected]

NA-fWebSOD

Clone the NA-fWebSOD repository:

# NA-fWebSOD=/path/to/clone/NA-fWebSOD
git clone https://github.com/shenyunhang/NA-fWebSOD.git $NA-fWebSOD
cd $NA-fWebSOD
git submodule update --init --recursive

Install Python dependencies:

pip3 install -r requirements.txt

Set up Python modules:

make

Build the custom C++ operators library:

./build_ops.sh

Dataset Preparation

Training Data

Download flickr_voc from this here and untar File:

tar xvf flickr_voc.tar
ln -s /path/to/clone/flickr_voc $NA-fWebSOD/detectron/datasets/data/flickr_voc

Download flickr_coco from this here and untar File:

tar xvf flickr_coco.tar
ln -s /path/to/clone/flickr_coco $NA-fWebSOD/detectron/datasets/data/flickr_coco

Download flickr_clean from this here and untar File:

tar xvf flickr_clean.tar
ln -s /path/to/clone/flickr_clean $NA-fWebSOD/detectron/datasets/data/flickr_clean

Testing Data

Please follow this to creating symlinks for PASCAL VOC.

Download MCG proposal from here to detectron/datasets/data, and transform it to pickle serialization format:

cd detectron/datasets/data
tar xvzf MCG-Pascal-Main_trainvaltest_2007-boxes.tgz
cd ../../../
python3 tools/convert_mcg.py voc_2007_train detectron/datasets/data/MCG-Pascal-Main_trainvaltest_2007-boxes detectron/datasets/data/proposals/mcg_voc_2007_train.pkl
python3 tools/convert_mcg.py voc_2007_val detectron/datasets/data/MCG-Pascal-Main_trainvaltest_2007-boxes detectron/datasets/data/proposals/mcg_voc_2007_val.pkl
python3 tools/convert_mcg.py voc_2007_test detectron/datasets/data/MCG-Pascal-Main_trainvaltest_2007-boxes detectron/datasets/data/proposals/mcg_voc_2007_test.pkl

Finnally, We have the following directory structure:

NA-fWebSOD
|_ detectron
|_ datasets
|_ data
|_ flickr_voc
|_ images
|_ images.json
|_ images.txt
|_ ...
|_ flickr_coco
|_ images
|_ images.json
|_ images.txt
|_ ...
|_ flickr_clean
|_ images
|_ images.json
|_ images.txt
|_ ...
|_ VOC2007
|_ coco
|_ ...

Model Preparation

Download models from this here and untar File:

tar xvf models.tar
mv models $NA-fWebSOD

Then We have the following directory structure:

NA-fWebSOD
|_ models
|  |_ VGG
|  |_ |_ VGG_ILSVRC_16_layers_v1.pkl
|_ ...

Quick Start: Using NA-fWebSOD

NA-fWebSOD

Flickr voc

./scripts/train_wsl.sh --cfg configs/flickr_voc/webly_wsddn_V-16-C5_1x.yaml OUTPUT_DIR experiments/webly_wsddn_v-16_flickr_voc_`date +'%Y-%m-%d_%H-%M-%S'`

Flickr clean

./scripts/train_wsl.sh --cfg configs/flickr_clean/webly_wsddn_V-16-C5_1x.yaml OUTPUT_DIR experiments/webly_wsddn_v-16_flickr_clean_`date +'%Y-%m-%d_%H-%M-%S'`

Flickr coco

./scripts/train_wsl.sh --cfg configs/flickr_coco/webly_wsddn_V-16-C5_1x.yaml OUTPUT_DIR experiments/webly_wsddn_v-16_flickr_coco_`date +'%Y-%m-%d_%H-%M-%S'`

na-fwebsod's People

Contributors

Stargazers

Watchers

Forkers

bityangke zyg11 younetcq xrosliang mammoth9 xychen9459 cv-ip torchlight-ljj sisrfeng

na-fwebsod's Issues

Do ground-truth labels mean BN or FL in the sentence? WD and RD have high responses to ground-truth labels and foreground label respectively.

Hi, would you mind explaining here a little more in detail?

We visualize the proposal scores and pixel gradient
maps of the WD and RD heads in Fig. 4. WD and RD have
high responses to ground-truth labels and foreground labels(FL), 
respectively. The two results combined from WD and
RD can decompose BN from FL.

Does decompose mean that if there is a bus and a car in the picture with lable bus, the model will ignore the region containing a car to make sure the label can match the picture?

Many thx!

foreground label means the your query while retrieving images from the Flickr, and background label means the tags provided by Flickr (user or machine) ?

If the foreground/background label is true:
    you refer to it as groud truth.
else:
    you refer to it as  foreground/background noise.

Noise is mainly from Flickr' s robot.

right?
Thx!

One-hot or multi-hot-encoding?

Hi, I find that in your dataset flickr-clean, each image contains only 1 category. In flickr-voc, some images contain 2 categories or more. In the latter case, should the label be multi-hot? For example, an image has two annotations [category_id:0] and [category_id:2]. Should the label be [1,0,1,……,0]?

ref:
如果一个样本属于多个类，比如在CelebA数据集中一张人脸照片是椭圆形的，戴眼镜的... 这个时候样本的label标注是multi-hot-encoding的，也就是会出现多个标签1。如[0,1,1,0]， 1代表属性出现，0代表属性不出现。
https://www.jianshu.com/p/09c49f74308a
https://stats.stackexchange.com/questions/467633/what-exactly-is-multi-hot-encoding-and-how-is-it-different-from-one-hot

Would you mind showing the visualization results of the code?

To compare the results with your baseline, I think visualization results are more straight forward.

./build_ops.sh编译问题

PLEASE FOLLOW THESE INSTRUCTIONS BEFORE POSTING

Please thoroughly read README.md, INSTALL.md, GETTING_STARTED.md, and FAQ.md
Please search existing open and closed issues in case your issue has already been reported
Please try to debug the issue in case you can solve it on your own before posting

After following steps 1-3 above and agreeing to provide the detailed information requested below, you may continue with posting your issue

(Delete this line and the text above it.)

Expected results

What did you expect to see?

Actual results

What did you observe instead?

Detailed steps to reproduce

E.g.:

The command that you ran

System information

Operating system: ?
Compiler version: ?
CUDA version: ?
cuDNN version: ?
NVIDIA driver version: ?
GPU models (for all devices if they are not all the same): ?
PYTHONPATH environment variable: ?
python --version output: ?
Anything else that seems relevant: ?

The AP on COCO seems a little too low. Have you tried other detection models?

What about Faster R-CNN instead of VGG ?

./build_ops.sh meet some error

Hi, would you mind giving me some help on this?

My command(as root user):
#my path of NA-fWebSOD is '/opt/data/private/webly'
#pytorch=~/pytorch
git clone https://github.com/pytorch/pytorch.git $pytorch
cd $pytorch
git checkout v1.3.0
git submodule update --init --recursive
pip install -r $pytorch/requirements.txt
cd $pytorch
USE_OPENCV=On USE_LMDB=On BUILD_BINARY=On python3 setup.py install
pip install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
pip install git+git://github.com/waspinator/[email protected]
git clone https://github.com/shenyunhang/NA-fWebSOD.git /opt/data/private/webly
cd /opt/data/private/webly
git submodule update --init --recursive
pip install -r requirements.txt
apt-get install libeigen3-dev
ln -s /usr/include/eigen3/Eigen /usr/include/Eigen
make

Cannot open the links to training data and model

The links cannot be open now. So we cannot get your training data and model. Would you please update these links?

Where is the implementation of bagging mixup?

Just cannot find...

shenyunhang / na-fwebsod Goto Github PK

na-fwebsod's Introduction

Noise-Aware Fully Webly Supervised Object Detection

Introduction

License

Citing NA-fWebSOD

Installation

Caffe2

Other Dependencies

NA-fWebSOD

Dataset Preparation

Training Data

Testing Data

Model Preparation

Quick Start: Using NA-fWebSOD

NA-fWebSOD

na-fwebsod's People

Contributors

Stargazers

Watchers

Forkers

na-fwebsod's Issues

PLEASE FOLLOW THESE INSTRUCTIONS BEFORE POSTING

After following steps 1-3 above and agreeing to provide the detailed information requested below, you may continue with posting your issue

Expected results

Actual results

Detailed steps to reproduce

System information

Recommend Projects

Recommend Topics

Recommend Org