PyTorch | Arxiv | Project

PyTorch implementation of our D2S model for zero-shot sketch-based image retrieval:
Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval
Sounak Dey, Pau Riba, Anjan Dutta, Josep Llados and Yi-Zhe Song
CVPR, 2019

• Linux (tested on Ubuntu 16.04)
• NVIDIA GPU + CUDA CuDNN
• 7z

sudo apt-get install p7zip-full

We took the first steps to move towards practical zero shot sketch based image retrieval systems (see the paper for more detail). To this end, we have used Quick Draw! to curate the sketches and for the images we would like to thanks Flickr API for such an amazing API.

The structure of this repo is as follows:

1. Installation
2. Getting the data
3. How to train models
4. At last how to test and evaluate

• Clone this repository

git clone https://github.com/sounakdey/doodle2search.git
cd doodle2search

• Install the requirements (not checked)

pip3 install -r requirements.txt

• Sketchy
• TU-Berlin

bash download_datasets.sh

• QuickDraw-Extended sketches and images

Finally we are ready to train. Magical words are:

python3 src/train.py sketchy_extended --data_path <mention the data path of the dataset>

The first argument is the dataset name, which you can replace it with tuberlin_extend or quickdraw_extend. You can check the options.py for changing a lot of the options such dimension size, different models, hyperparameters, etc.

python3 src/test.py sketchy_extend --data_path <mention the data path of the dataset> --load <path of the trained models>

@InProceedings{Dey_2019_CVPR,
    author = {Dey, Sounak and Riba, Pau and Dutta, Anjan and Llados, Josep and Song, Yi-Zhe},
    title = {Doodle to Search: Practical Zero-Shot Sketch-Based Image Retrieval},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June},
    year = {2019}
}

Thank you and sorry for the bugs!

• Sounak Dey (@SounakDey)
• Pau Riba (@PauRiba)
• Anjan Dutta (@AnjanDutta)