hukkelas / deep_privacy2 Goto Github PK

View Code? Open in Web Editor NEW

278.0 6.0 31.0 13.69 MB

DeepPrivacy2 - A Toolbox for Realistic Image Anonymization

License: Apache License 2.0

Python 93.49% C++ 1.92% Cuda 4.33% Dockerfile 0.25%

deep_privacy2's Introduction

DeepPrivacy2 - A Toolbox for Realistic Image Anonymization

[Paper] [Appendix] [Video Demo] [Documentation]

DeepPrivacy2 is a toolbox for realistic anonymization of humans, including a face and a full-body anonymizer.

DeepPrivacy first detects, then recursively anonymization all individuals in an image with a Generative Adversarial Network (GAN) that synthesizes one individual at a time.

Published Papers

This repository provide source code for the following papers

[PDF] [Appendix] [Conference Presentation] DeepPrivacy2: Towards Realistic Full-Body Anonymization. Published at WACV 2023.
[PDF] [Appendix] Does Image Anonymization Impact Computer Vision Training? Published at CVPR Workshop on Autonomous Driving 2023.
[PDF] [Appendix] Synthesizing Anyone, Anywhere, in Any Pose.

DeepPrivacy2 vs DeepPrivacy1

This repository improves over the original DeepPrivacy repository with the following new features:

Full-body anonymization: Anonymize the entire human body with a single generator
Improved Face Anonymization: Improved quality and higher resolution (256x256 vs. 128x128) face anonymization without relying on facial landmark detection.
Attribute Guided Anonymiation: Anonymize faces guided on text prompts using StyleMC - [Video Demo].
Code cleanup and general improvements: Extensive refactoring, bugfixes, and improvements yielding improved results and faster training.

Useful Links

Quick Start

Installation

We recommend to setup and install pytorch with anaconda following the pytorch installation instructions.

Clone repository: git clone https://github.com/hukkelas/deep_privacy2/.
Install using setup.py:

pip install -e .

or:

pip install git+https://github.com/hukkelas/deep_privacy2/

See the documentation for more installation instructions.

Anonymization

anonymize.py is the main script for anonymization.

Full-Body Anonymization

python3 anonymize.py configs/anonymizers/FB_cse.py -i media/regjeringen.jpg --output_path output.png --visualize

Face Anonymization

python3 anonymize.py configs/anonymizers/face.py -i media/regjeringen.jpg --output_path output.png --visualize

Webcam anonymization

python3 anonymize.py configs/anonymizers/FB_cse.py --webcam

See the documentation for more detailed instructions for anonymization.

Gradio Demos

The repository includes gradio demos to show of the capabilities of DeepPrivacy2.

Face anonymization. Test it on Hugging Face.

python3 -m gradio_demos.face

Full-body anonymization. Test it on Hugging Face.

python3 -m gradio_demos.body_cse

License

This repsitory is released under Apache 2.0 License, except for the following:.

Code under sg3_torch_utils/. This code is modified from github.com/NVlabs/stylegan2-ada-pytorch. Separate license is attached in the directory.
Detection network: See Detectron2 License.
All checkpoints follow the license of the datasets. See the respective datasets for more information.
Code under dp2/detection/models/vit_pose. This code is modified from https://github.com/gpastal24/ViTPose-Pytorch, where code is adapted from OpenMMLab. Original license is Apache 2-0.

Citation

If you find this repository useful, please cite:

@inproceedings{hukkelas23DP2,
  author={Hukkelås, Håkon and Lindseth, Frank},
  booktitle={2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)}, 
  title={DeepPrivacy2: Towards Realistic Full-Body Anonymization}, 
  year={2023},
  volume={},
  number={},
  pages={1329-1338},
  doi={10.1109/WACV56688.2023.00138}}

deep_privacy2's People

Contributors

Stargazers

Watchers

deep_privacy2's Issues

No module names "tops"

Hello dear Hukkelas,

when i try to use deep privacy 2 for the webcam with this command line :
python3 anonymize.py configs/anonymizers/FB_cse.py --webcam

I receive this output :
ModuleNotFoundError: No module named 'tops'

Can you help me with this ??

Error downloading object

This states that this repository is over its data quota.

Issues building detectron2

Dear Håkon and Frank,

i am having issues setting up deep_privacy2.
With my current setup, i cannot compile detectron2.
my system: Linux Mint 21, Python 3.10, using a virtualenv with pip3 ver. 22.3.1
g++ version: 11.3.0.

Error: nvcc gives the following error:

box_iou_rotated_cuda.cu -o /tmp/pip-install-ivei5q55/detectron2_6d6f876c8cab4856b7bf43bd6663c37d/build/temp.linux-x86_64-3.10/tmp/pip-install-ivei5q55/detectron2_6d6f876c8cab4856b7bf43bd6663c37d/detectron2/layers/csrc/box_iou_rotated/box_iou_rotated_cuda.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr --compiler-options ''"'"'-fPIC'"'"'' -O3 -DCUDA_HAS_FP16=1 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ -DTORCH_API_INCLUDE_EXTENSION_H '-DPYBIND11_COMPILER_TYPE="_gcc"' '-DPYBIND11_STDLIB="_libstdcpp"' '-DPYBIND11_BUILD_ABI="_cxxabi1011"' -DTORCH_EXTENSION_NAME=_C -D_GLIBCXX_USE_CXX11_ABI=0 -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -std=c++14 /usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’: 435 | function(_Functor&& __f)

do you have an idea how i can fix the issue; or could you maybe provide a Dockerfile for deep_privacy2?
any help is highly appreciated!

How to reproduce the keypoint detection and Re-identification results in your paper?

Hello @hukkelas ,
Your repository is outstanding! I was wondering if you could provide detailed instructions on how to replicate the settings and experiments outlined in your paper, particularly those related to human keypoint detection and re-identification. Your guidance would be greatly appreciated!

Is the TRACK function working properly?

Thanks for sharing the great code.

By the way, is the track function working correctly?
When using anonymize.py to anonymize a video, it appears that the same person is not anonymized in the same style even if track is specified as an argument.
Specifically, even if I specify track as an argument, all_styles=None, so the tracking results do not seem to be reflected in the style.
I would appreciate it if you could tell me how this should be corrected.

face demo problem

seems like the demo with face configuration does not allow the upload of images, but tries to access the webcam in both modalities .

Running in Google Colab: Output and Running Time Issues

Hi @hukkelas,

Thanks for your work. I am currently running DeepPrivacy2's face anonymization in Google Colab using the V100 GPU, specifically the face.py anonymizer on a subset of the FDF256.

Google Colab's Parameters:

Python Version: 3.10
TensorFlow version: 2.15.0
CUDA ver. 12.2
Pytorch Version: 2.2.1+cu121

When I try to run anonymization on an image

! python3 anonymize.py configs/anonymizers/face.py -i ../DeepPrivacy2_Data/fdf256_DeepPrivacy2/002001.png --output_path ../002001.png

I get the following output:

/content/deep_privacy2
2024-03-17 14:04:53.462022: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2024-03-17 14:04:53.462077: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2024-03-17 14:04:53.463799: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-03-17 14:04:55.284089: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
pygame 2.5.2 (SDL 2.28.2, Python 3.10.12)
Hello from the pygame community. https://www.pygame.org/contribute.html
error: XDG_RUNTIME_DIR not set in the environment.
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5701:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM default
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5178:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5701:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2664:(snd_pcm_open_noupdate) Unknown PCM default
Saving outputs to: outputs/anonymizers/face
2024-03-17 14:05:00,615 [INFO ] logger.py: 102 [0]
anonymizer = dp2.anonymizer.Anonymizer(
    _args_=[],
    detector=dp2.detection.face_detector.FaceDetector(
        _args_=[],
        cache_directory=PosixPath("outputs/face_detection_cache"),
        face_detector_cfg={"clip_boxes": True, "name": "DSFDDetector"},
        face_post_process_cfg={"fdf128_expand": False, "target_imsize": [256, 256]},
        score_threshold=0.3,
    ),
    face_G_cfg="configs/fdf/stylegan.py",
)
checkpoint_dir = PosixPath("outputs/anonymizers/face/checkpoints")
common.experiment_name = None
common.logger_backend = ["wandb", "stdout", "json", "image_dumper"]
common.output_dir = PosixPath("outputs")
common.wandb_project = "deep_privacy2"
detector = dp2.detection.face_detector.FaceDetector(
    _args_=[],
    cache_directory=PosixPath("outputs/face_detection_cache"),
    face_detector_cfg={"clip_boxes": True, "name": "DSFDDetector"},
    face_post_process_cfg={"fdf128_expand": False, "target_imsize": [256, 256]},
    score_threshold=0.3,
)
experiment_name = "configs/anonymizers/face.py"
filename = "configs/anonymizers/face.py"
output_dir = PosixPath("outputs/anonymizers/face")

2024-03-17 14:05:01,115 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
  warnings.warn(

2024-03-17 14:05:01,115 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=None`.
  warnings.warn(msg)

Saving outputs to: outputs/fdf/stylegan
2024-03-17 14:05:04,332 [INFO ] logger.py: 102 [0]Generator loaded, num parameters: 43.131523M
2024-03-17 14:05:04,333 [INFO ] logger.py: 102 [0]W cluster centers loaded. Number of centers: 512
2024-03-17 14:05:04,381 [INFO ] logger.py: 102 [0]Loaded generator from: configs/fdf/stylegan.py
2024-03-17 14:05:04,382 [WARNI] logger.py: 102 [0]Overwriting previous file: ../002001.png
2024-03-17 14:05:05,560 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,592 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,593 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,596 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,737 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,738 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,740 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,747 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,752 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,763 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,769 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,773 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,783 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,787 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,791 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,801 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

2024-03-17 14:05:05,806 [WARNI] warnings.py: 109 /usr/local/lib/python3.10/dist-packages/torch/nn/modules/instancenorm.py:80: UserWarning: input's size at dim=1 does not match num_features. You can silence this warning by not passing in num_features, which is not used because affine=False
  warnings.warn(f"input's size at dim={feature_dim} does not match num_features. "

Saved to: ../002001.png

I still get an anonymised output, but it takes 17 seconds to complete.

As such I have a couple of questions:

How can I shorten the running time for a single anonymization, given if I were to measure FID, I would require more than 2000 samples?
You suggest face.py for face anonymization only, but can FB_cse.py also work with the FDF256 in a similar runtime?
What is the pygame terminal message, where is it from, and is it necessary to runtime?
What is the reason for the ASLA errors, given that I do have access to a GPU in Google Colab?
Why does it save to outputs/anonymizers/face and outputs/fdf/stylegan if I am only running anonymization and not training?
How can I silence the UserWarning: input's size at dim=1 does not match num_features in-code?

Thanks for taking a look.

Kind Regards.

Weights for the model

Hi Hakon,

Thank you for your repo. I found your work very curious to understand how it will work on my own pictures.
How can I test your algorithm? Are there any weights for your model?

Can't install Deep_privacy2 in the Google Collab with miniconda and python3.9

Hello all,

I was given to implement deep_privacy2 anonymization in Google Colab.
After successfully installing miniconda and pytorch, I run the following code:
pip install git+https://github.com/hukkelas/deep_privacy2/

Result

As a n output I got following error:

Looking forward for you help!

Black Mask

Is there a way to mask the full body with a black silhouette? Kind of like the --vd option without the transparency and frame.

Cant open camera by index

I have installed it with docker. It works fine on static images. However when I do webcam anonymization, I get "cant open camera by index".

I have tried built in camera and also usb camera both are giving same error.

I have also tried to change the index here

deep_privacy2/anonymize.py

Line 158 in 50ce66a

cap = BufferlessVideoCapture(0, width=1920, height=1080)

from 0-2 but the problem remains.

When i run a dummy open camera on a separate python file using opencv it works fine.

OpenCV error when using docker

Hi,

unfortunately, I get the following error when using your dockerfile to build a container and test the code:
AttributeError: module 'cv2.dnn' has no attribute 'DictValue'

I think it is an issue with the opencv version, since installing a newer version inside the container resolves it.

MPS on MacOS?

Before I start digging through the code, is there a straightforward way to use MPS for M1/M2 Macs rather than CUDA for NVIDIA? PyTorch does support MPS: https://pytorch.org/docs/stable/notes/mps.html

ERROR: "ModuleNotFoundError: No module named 'tops'"

Traceback (most recent call last): File "anonymize.py", line 4, in <module> import tops ModuleNotFoundError: No module named 'tops'

SSL Certificate issue on Mac

Hi, I have followed all the steps for installation and when I try to run the anonymizer I get an SSL Certificate issue: urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>

requirements.txt

It would be great if you can provide the requirements.txt you used to install all the required packages. Thank you!

'MultiObjectTracker' object has no attribute 'detections_matched_ids'

When I try to use --track flag to run anonymize.py I get the error saying that the object of MultiObjectTracker class doesn't have the attribute named 'detections_matched_ids'. From the motpy pypi package I installed, I checked the class definition for MultiObjectTracker class and it doesn't seem to have this attribute in it. The attribute 'detections_matched_ids' is referred in forward() function of Anonymizer class to retrieve the track_ids. But the attribute seems to be absent everywhere in the class definition of MultiObjectTracker class.

Whereas, I checked this script (https://github.com/wmuron/motpy/blob/master/motpy/tracker.py) from the motpy repository and they seem to have it defined as an attribute.

My question: Was the attribute 'detections_matched_ids' intentionally removed in this project repository? I feel it could be a mistake since you try to use the track_ids in Anonymizer class. Let me know if I'm missing out something! Thanks!

Installation on Windows

I ran into a problem trying to install deep_privacy2 on Windows. Seems like detectron can't be installed on Windows without changes to the package: facebookresearch/detectron2#1601
Windows is not officially supported by detectron.

Maybe a hint regarding supported OS in the guide https://www.hukkelas.no/deep_privacy2/#/README would be helpful to other users?

ModuleNotFoundError: No module named 'tops'

Getting the following error while executing the anonymize.py script.

Traceback (most recent call last):
File "/content/deep_privacy2/anonymize.py", line 2, in
import tops
ModuleNotFoundError: No module named 'tops'

I have already browsed through the repository, but couldn't find any module called tops.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

hukkelas / deep_privacy2 Goto Github PK

deep_privacy2's Introduction

DeepPrivacy2 - A Toolbox for Realistic Image Anonymization

Published Papers

DeepPrivacy2 vs DeepPrivacy1

Useful Links

Quick Start

Installation

Anonymization

Gradio Demos

License

Citation

deep_privacy2's People

Contributors

Stargazers

Watchers

Forkers

deep_privacy2's Issues

Result

Recommend Projects

Recommend Topics

Recommend Org