hqqasw / person-search-ppcc Goto Github PK

person search by progressive propagation via competitive consensus

Python 100.00%

person-search-ppcc's Introduction

Person Search by Progressive Propagation via Competitive Consensus (PPCC)

This is the implement of our ECCV 2018 paper

Person Search in Videos with One Portrait Through Visual and Temporal Links.
Qingqiu Huang, Wentao Liu, Dahua Lin. ECCV 2018, Munich.

This project is based on our person search dataset -- Cast Search in Movies (CSM) . More details about this dataset can be found in our project page.

Basic Usage

Download the affinity matrices and meta data of CSM from Google Drive or Baidu Wangpan
Put affnity matrix in "**/data/affinity" and meta data in "**/data/meta". Here "**" means the path that you clone this project to.
Run "matching.py" for visual matching and "propagation.py" for lable propagation. Example:
python propagation.py --exp in --gpu_id -1 --temporal_link

More Details

The downloaded affinity matrices are calculate by the consin simmilarity of the visual features bewteen the instances. More specific, we use face features for cast-tracklet links and body features for tracklet-tracklet links. The face model is a Resnet-101 trained on MS-Celeb-1M. The body model is a Resnet-50 pretrianed on ImageNet and finetune on the training set of CSM. You can also train your own model on CSM.
We implement both CPU and GPU version of PPCC, you can choose any one of them by setting the paprameter "gpu_id" (-1 for CPU and others for a specific GPU). The GPU code is based on PyTorch. You are recommand to use GPU version since it is much faster, especially for the "ACROSS" experiment settting.

Citation

@inproceedings{huang2018person,
    title={Person Search in Videos with One Portrait Through Visual and Temporal Links},
    author={Huang, Qingqiu and Liu, Wentao and Lin, Dahua},
    booktitle={Proceedings of the European Conference on Computer Vision (ECCV)},
    pages={425--441},
    year={2018}
}

person-search-ppcc's People

Contributors

Stargazers

Watchers

person-search-ppcc's Issues

If the portraits are unlabeled data , does the algorithm still work?

@hqqasw Dear Qingqiu,

after reading your paper, I have two questions below:

1, If the portraits are unlabeled data, even the gallery set is also unlabeled, Can the algorithm proposed still work? how?

2, If I want to search a portrait doesn't include in the gallery set, how to set the initial probability? should the length of one-hot vector be C+1 ?

face and body?

Thank you very much for your contribution.

After I read the paper, I was a bit confused. What kind of form does the face and body correspond to? I am currently downloading CSM on Baidu's network disk, which is really slow.
After I looked at propagation.py, the face and body matrices, did you train them on the CSM, and then took their correspondence out as input to the label propagation algorithm?

Sorry to disturb you again, I just ran a bit of code, just a simple result, I would like to ask, is there such a test demo, such as I enter the photo of Andy Lau, and then gamble in the movie to find him Photo?

Use a video of my own, and then can I use an image to retrieve the shot at which time it appeared

after reading your paper, I have a question below:
Use a video of my own, and then can I use an image to retrieve the shot where he appears?

Could you please release the face/body visual feature or the training code for the cast, and the tracklet?

Could you please release the face/body visual feature or the training code for the cast, and the tracklet?
Thanks.

Script for extracting features

Hi I am Wiktor,
I really like your project and I would like to extract body features from a custom image dataset using your model.
Could you specify what should be put as args in data_root and list_file in your feature extraction script?
Thank!

Could you share the CSM pretrained IDE baseline model?

Hi Qingqiu,

Thanks for sharing your great work!

I'm wondering if you can share your CSM pretrained weights for IDE re-id baseline or if you can reference me which IDE repo you were using for fine-tuning on CSM dataset.

Thanks in advance!

Chundi

what is the face feature if the face detector do not detect any face in the images?

Hi,
I would like to reproduce your work with my own dataset. First, I need to use the face detector to get the location of the face from cast or tracklet image, and if it can't find any face from the image, what is the face feature for that tracklet?

Now I am using the Restnet50 and Resnet101 to extract the body and face feature, and then using the linear layer to downsample the features to 256 dimensions, is that correct? Then, I would use the feature to calculate the cosine similarity for both face to trcaklet and tracklet to tracklet. Finally, using the similarity score to run your code, is that the correct way to reproduce the work. Thank you so much.

Could you provide face classifier pretrained weight?

Hi,
I would like to train my face feature extractor to reproduce this work, but the download link of the Celeb-1M dataset has been deleted. So could you please provide the pre-trained weight of the face classifier.
Thank you so much