Giter VIP home page Giter VIP logo

khaclinh / pp4av Goto Github PK

View Code? Open in Web Editor NEW
10.0 1.0 1.0 1.46 MB

PP4AV: A benchmarking Dataset for Privacy-preserving Autonomous Driving

Home Page: https://openaccess.thecvf.com/content/WACV2023/html/Trinh_PP4AV_A_Benchmarking_Dataset_for_Privacy-Preserving_Autonomous_Driving_WACV_2023_paper.html

License: MIT License

Python 8.73% MATLAB 0.62% Makefile 0.01% Jupyter Notebook 88.91% Cython 0.57% C++ 0.65% C 0.52%
autonomous-driving privacy-preserving

pp4av's Introduction

PP4AV: A benchmarking Dataset for Privacy-preserving Autonomous Driving

Quick start

News

2022.10:

2022.7: PP4AV v1.0 is released with images, face and license plate bounding box annotations.

Prerequisites

The code of baseline model and auxiliary scripts is built with following libraries:

  • Python >= 3.6, <3.9
  • Pillow = 8.4.0 (see here)
  • loguru==0.5.3
  • matplotlib==3.3.4
  • numpy==1.19.5
  • opencv_python==4.5.3.56
  • PyYAML==6.0
  • recommonmark==0.7.1
  • setuptools==58.0.4
  • Sphinx==3.2.1
  • sphinx_rtd_theme==0.5.0
  • tabulate==0.8.7
  • thop==0.0.31.post2005241907
  • tqdm==4.31.1
  • tensorboard==2.3.0

Dataset

Data Summary

PP4AV is the first public dataset with faces and license plates annotated with driving scenarios. P4AV provides 3,447 annotated driving images for both faces and license plates. For normal camera data, we sampled images from the existing videos in which cameras were mounted in moving vehicles, running around the European cities. The images in PP4AV were sampled from 6 European cities at various times of day, including nighttime. We use the fisheye images from the WoodScape dataset to select 244 images from the front, rear, left, and right cameras for fisheye camera data. PP4AV dataset can be used as a benchmark suite (evaluating dataset) for data anonymization models in autonomous driving.

Dataset description

The detail of dataset collection, structure, annotation, format are described in Hugging Face PP4AV dataset. You also can check the description of PP4AV dataset in this document.

Download

Dataset Manipulation

We profile the utility scripts for manipulating the PP4A dataset. Please check this document for detail of guidance.

Dataset License

Creative Commons License
This PP4AV dataset is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Baseline model and performance

conda create --name pp4av-env python=3.8
conda activate pp4av-env
pip3 install -U pip && pip3 install -r requirements.txt
pip3 install torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
pip3 install -v -e .  # or  python3 setup.py develop
  • Model performance reports
    Method Normal images Fisheye images
    AP_50 AR_50 AP_50 AR_50
    Face UAI Anonymizer 42.64% 83.70% 43.98% 53.33%
    AWS API 63.69% 73.33% 40.72% 46.67%
    Google API 7.97% 8.99% 7.64% 8.89%
    RetinaFace 62.71% 88.28% 43.82% 62.96%
    Yolo5Face 69.31% 93.96% 69.59% 82.96%
    PP4AV 76.22% 92.52% 59.20% 63.92%
    License plate ALPR 38.79% 41.68% 17.26% 31.21%
    Nvidia LPDnet 57.41% 58.44% 24.90% 26.24%
    UAI Anonymizer 84.89% 85.61% 44.14% 53.90%
    PP4AV 88.12% 91.88% 49.53% 58.17%

Evaluation

  • We provide scripts to evaluate models in both WIDER FACE and standard evaluative methods for object detection. You can follow up the WIDER FACE evaludation document for plotting the PR-curve.

  • We also provide the evaluation script, which includes performance metrics such as Average Recall, Average Precision, and qualitative analysis. This evaluation's document and coding script are available at [this document and coding] (evaluations/baseline_evals/README.md).

  • The script for comprehensive analysis is available in this section.

Citation

If you think this work is useful for you, please cite

@article{PP4AV2022,
  title = {PP4AV: A benchmarking Dataset for Privacy-preserving Autonomous Driving},
  author = {Linh Trinh, Phuong Pham, Hoang Trinh, Nguyen Bach, Dung Nguyen, Giang Nguyen, Huy Nguyen},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year = {2023}
}

FAQ

TBD

Acknowledgements

The baseline model is based on YOLOX. The data annotation tool was done with CVAT too.

Contact

If you have any problems about PP4AV, please contact Linh Trinh at [email protected].

pp4av's People

Contributors

khaclinh avatar

Stargazers

Andrei Moraru avatar Valfride Nascimento avatar Nguyễn Đức Anh avatar  avatar  avatar Nguyen Van Linh avatar  avatar  avatar  avatar Rayson Laroca avatar

Watchers

 avatar

Forkers

fsoft-aic

pp4av's Issues

Run evaluation on PP4AV with same approach as evaluation results in the paper

Hi,

I have read the research paper, and evaluated license & face model on PP4AV dataset and didn't get matching results, AP/AR for face models not even close to 40%. I was using https://github.com/rafaelpadilla/review_object_detection_metrics for the evaluation.

I did not find a way to evaluate custom model using your filtering approach for smaller bounding boxes. Is there a script that is usable to run against PP4AV GT detections in format (YOLO or COCO) and get results AP/AR in the same way you have shown in the research paper to produce results that can be compared.

Thank you for research and creating the first step towards a good benchmarking dataset for anonymization in the traffic environment.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.