orb-hd / deface Goto Github PK
View Code? Open in Web Editor NEWVideo anonymization by face detection
License: MIT License
Video anonymization by face detection
License: MIT License
I've been experimenting with this tool, it works amazing. I'm testing deface to run on jpg images downloaded from a webcam.
I noticed quite a file size increase on these jpg's:
Original: 435kb
Deface: 610kb - without any faces in the picture
Although it doesn't look like much, I'm planning on using this on quite a scale, which makes the total increased data storage quite large and I would like to avoid that where possible.
Any ideas what might increase the file size, even without faces on them?
Would you mind giving an example of the arguments for this option? even the default option doesn't seem to work.
I was looking into this project and was wondering if it would be possible to select specific faces out of the file to leave unblurred.
This is a great tool, but unfortunately we cannot incorporate it in our project. The video process duration is prohibitive ( FHD video, typically 1 hour long).
Suggestion: since in our videos, faces appear rarely and for a few seconds, it would be great if an option existed where we could mark the time span that the faces appear, so the tool would process just a few seconds and not the entire video. This would significantly speed up the process.
Thanks for your time.
Running in docker.
deface.Dockerfile
:
FROM docker.io/ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get -yq update && apt-get -yq install \
software-properties-common \
python3 \
python3-pip \
git \
cmake pkg-config mesa-utils libglu1-mesa-dev freeglut3-dev mesa-common-dev libglew-dev libglfw3-dev libglm-dev libao-dev libmpg123-dev
RUN pip3 install pip
#RUN pip3 install deface
RUN pip3 install 'git+https://github.com/ORB-HD/deface'
WORKDIR /app
deface.sh
:
#!/bin/bash
docker build -f deface.Dockerfile -t deface .
docker run \
-it \
--rm \
-v $(pwd):/app \
--entrypoint=/bin/bash \
deface
Start container and enter it:
./deface.sh
Then run deface
on friends.png
image:
root@ed084ad5d8a4:/app# deface friends.png
Input: friends.png
Output: friends_anonymized.png
[ERROR:[email protected]] global /io/opencv/modules/dnn/src/net_impl.cpp (1168) getLayerShapesRecursively OPENCV/DNN: [Convolution]:(onnx_node_output_0!363): getMemoryShapes() throws exception. inputs=1 outputs=0/1 blobs=1
[ERROR:[email protected]] global /io/opencv/modules/dnn/src/net_impl.cpp (1174) getLayerShapesRecursively input[0] = [ 1 4 1056 1600 ]
[ERROR:[email protected]] global /io/opencv/modules/dnn/src/net_impl.cpp (1182) getLayerShapesRecursively blobs[0] = CV_32FC1 [ 32 3 3 3 ]
[ERROR:[email protected]] global /io/opencv/modules/dnn/src/net_impl.cpp (1184) getLayerShapesRecursively Exception message: OpenCV(4.6.0) /io/opencv/modules/dnn/src/layers/convolution_layer.cpp:404: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 4 in function 'getMemoryShapes'
Traceback (most recent call last):
File "/usr/local/bin/deface", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/deface/deface.py", line 372, in main
image_detect(
File "/usr/local/lib/python3.10/dist-packages/deface/deface.py", line 187, in image_detect
dets, _ = centerface(frame, threshold=threshold)
File "/usr/local/lib/python3.10/dist-packages/deface/centerface.py", line 86, in __call__
heatmap, scale, offset, lms = self.net.forward(self.onnx_output_names)
cv2.error: OpenCV(4.6.0) /io/opencv/modules/dnn/src/layers/convolution_layer.cpp:404: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 4 in function 'getMemoryShapes'
Hi, I've been using this tool for quite a while now, thanks!
I upgraded my server to Ubuntu 22.04 today and it started failing after that. I'm not an expert in Pyhton, so having trouble debugging.
I did try to delete and then install the deface packages again, this does not seem to have any effect.
This is whats happening:
root@server:[/home/user] deface 21.jpg
Input: 21.jpg
Output: 21_anonymized.jpg
Traceback (most recent call last):
File "/usr/local/bin/deface", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.10/dist-packages/deface/deface.py", line 368, in main
image_detect(
File "/usr/local/lib/python3.10/dist-packages/deface/deface.py", line 183, in image_detect
dets, _ = centerface(frame, threshold=threshold)
File "/usr/local/lib/python3.10/dist-packages/deface/centerface.py", line 86, in __call__
heatmap, scale, offset, lms = self.net.forward(self.onnx_output_names)
cv2.error: OpenCV(4.7.0) /io/opencv/modules/dnn/src/layers/fast_convolution/winograd_3x3s1_f63.cpp:147: error: (-215:Assertion failed) _FX_WINO_IBLOCK == 3 && _FX_WINO_KBLOCK == 4 in function '_fx_winograd_accum_f32'
munmap_chunk(): invalid pointer
This is a great tool, thanks for the good work!
However, while I am able to use it to deface videos (mp4) on my local machine (Windows 10), it only works for images but not videos when I try to use it remotely on a Linux-based super-computing cluster. I get the following error when I try it on a video named "cut.mp4":
"Could not open file cut.mp4 as a video file with imageio. Skipping file..."
I have tried updating numpy, imageio, scikit-image, tqdm, and imageio-ffmpeg and it still did not work. I appreciate it if you could help me figure this out. Thanks!
I apologize if this has already been looked at, but would there be a way to add the ability to use a Coral TPU(USB) in place of Nvidia Cuda to speed up processing?
Sometimes, the video contains false positives popping in and out, in a scene which is otherwise not changing.
Other part of the scene is changing, and contains false negatives.
It would be nice if this scene separation could be hinted to deface in a form of a grayscale mask, where black would decrease the score of a face detected on that coordinate, and white would increase.
Traceback (most recent call last):
File "/home/greon/.local/bin/deface", line 8, in
sys.exit(main())
File "/home/greon/.local/lib/python3.8/site-packages/deface/deface.py", line 383, in main
image_detect(
File "/home/greon/.local/lib/python3.8/site-packages/deface/deface.py", line 191, in image_detect
anonymize_frame(
File "/home/greon/.local/lib/python3.8/site-packages/deface/deface.py", line 82, in anonymize_frame
draw_det(
File "/home/greon/.local/lib/python3.8/site-packages/deface/deface.py", line 42, in draw_det
blurred_box = cv2.blur(
cv2.error: OpenCV(4.7.0) /io/opencv/modules/imgproc/src/box_filter.dispatch.cpp:446: error: (-215:Assertion failed) !_src.empty() in function 'boxFilter'
In addition to the error, there is a problem that the position of the blur does not settle properly on the face if it is not a 4:3 ratio image. How can I solve this?
Hi!
How can I have the output file contain the audio that was in the input file?
I've installed the latest version from the github, so I could use the fps option. I did something simplistic like:
deface plain01.2125294.20220419131915.avi --ffmpeg-config '{"codec": "libx264","fps": 100}'
Which did produce a video with 100fps, however it is sped up, instead of having all the frames from the original video.
What I think is happening, is that the reader is not reading all the 100fps and when playing and this is why.
So I put a print(meta)
statement and it is in fact not read as the appropriate fps (in my case it was being read as 25fps, even though the video is 100fps).
I did a very quick fix which is available in my fork :
https://github.com/mysablehats/deface
It worked for me for 60, 100 and even 120 fps (it was running out of frames, but it still managed to produce a playable video with the same duration).
Hope this helps, cheers!
I have limited technical knowledge
I installed correctly in a cmd in windows (I run it as admin)
after that "deface" is not recognized as a command. Maybe its as easy as setting some directory to PATH (or alternatively typing the whole path to where it is).
I tried "pip where deface" ... says its not there
I know it was installed to: "C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface"
I tried getting into this directory and typing "deface". It also does not find it
then I tried " python3 deface.py" this is the output:
PS C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface> python3 deface.py D:\Downloads\villalba.mp4
Traceback (most recent call last):
File "C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface\deface.py", line 17, in
from deface import version
File "C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface\deface.py", line 17, in
from deface import version
ImportError: cannot import name 'version' from partially initialized module 'deface' (most likely due to a circular import) (C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface\deface.py)
PS C:\users\fernando\appdata\local\packages\pythonsoftwarefoundation.python.3.12_qbz5n2kfra8p0\localcache\local-packages\python312\site-packages\deface>
I don't know what else to try on windows. I will try to install a virtual ubuntu and try there, but it would be nice if there was a fool-proof way to easily use this tool in windows
Thank you for your efforts!
Hi, how to fix this problem?
Python 3.8.10
Ubuntu 20.04
Model parameter set to default
Hi, thanks for sharing your project. Is it possible to deface images with this library without having to change the source code? Thanks.
Hi,
Thanks for sharing such a good work.
While testing deface, I found degradation of image quality in the output images (even without any face).
(For example, input_image.png 917.3kB => input_image_anonymized.png 758.4kB)
Is there an optional argument or something to let the image quality same in the output image?
Thank you.
This is an issue I faced but was able to resolve by following a few steps documented below. Creating an issue here so that it can be documented by the owners somewhere if desired (or I can create a PR later with an FAQ section in the README if the owners prefer):
This is not an issue with the deface package, but rather the imageio
dependency. When running deface video.mp4
, I got the following error:
➜ deface video.mp4
Input: video.mp4
Output: video_anonymized.mp4
Could not open file video.mp4 as a video file with imageio. Skipping file...
OS: macOS Ventural 13.5.2
Chip: Apple M1 Pro
Python Version: Python 3.11.5
Homebrew: 4.1.12
I was able to resolve this by doing 2 things:
librist
and mbeditis
because somehow they weren't found by imageio, and then re-installing ffmpeg
(source)➜ brew uninstall librist --ignore-dependencies
➜ brew uninstall mbedtls --ignore-dependencies
➜ brew reinstall ffmpeg
imageio
expected env var IMAGEIO_FFMPEG_EXE
to be set to the ffmpeg path:➜ export IMAGEIO_FFMPEG_EXE="/opt/homebrew/Cellar/ffmpeg/6.0.1/bin/ffmpeg"
(source)
After this, running deface
worked for me as expected. Thanks for creating this! (Result here)
I'm anonymising long interview videos and so the time is limiting.
To improve this could you just use tracking, like is used in other object detection situations where you perform detection periodically and then tracking in the intermediate frames.
This should help in a couple of different ways:
Other optimisations
Number of expected faces:
could include a parameter for the number of faces in the image. If you're just expecting 2-3 then as long as the track is good on those then you can have less frequent detection, again speeding up performance.
Minimum face size:
This could help if you're mainly looking for people in the foreground and help reduce false positives.
On Mac with 1920x1080 webcam, "deface cam" will freeze on first frame. When using an older Mac with 1280x720 webcam no issue there.
There is a fix for frame grabbing using imageio.imiter, but I am not sure how to fix for the case of imageio.imread:
For example, this line will fail on 1920x1080 webcam:
for idx, frame in enumerate(iio.imiter("<%video0%>")):
This will pass on 1920x1080 webcam:
for idx, frame in enumerate(iio.imiter("<%video0%>", size=(1280, 720))):
Please help.
Hi! I'm using a macOS Ventura 13.5.2 with Apple M1 Pro to anonymize some videos but, I don't know why, when the process finish all the videos are in high-speed.
Hi, I understand the project is obviously targeted at blurring/removing face from images/videos.
I was wondering if doing the same to license plates could be included in this project?
Pictures and videos taken of a street often include cars, mopeds and such. Being able to blur/remove those as well, would be a nice addition
Cheers,
Thibault
I'm running this on Google Colab
!pip install deface
!pip install onnx==1.15.0
!pip install onnxruntime-gpu==1.16.2
!pip install tensorrt==8.6.1.post1
!deface /content/gdrive/MyDrive/input_video/a.mp4 --replacewith mosaic --thresh 0.4 -o /content/gdrive/MyDrive/input_video/a_blur.mp4
EP Error /onnxruntime_src/onnxruntime/core/session/provider_bridge_ort.cc:1193 onnxruntime::Provider& onnxruntime::ProviderLibrary::Get() [ONNXRuntimeError] : 1 : FAIL : Failed to load library libonnxruntime_providers_tensorrt.so with error: libnvinfer.so.8: cannot open shared object file: No such file or directory
when using ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'AzureExecutionProvider', 'CPUExecutionProvider']
Falling back to ['CUDAExecutionProvider', 'CPUExecutionProvider'] and retrying.
Running on CUDAExecutionProvider.
PS Z:\> deface VID_20220608_185133.mp4 --thresh 0.02
2022-06-09 00:23:37.7344718 [E:onnxruntime:Default, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "d:\program files\python\python39\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
An implementation of skipping frames in which the expensive computing of centerface gets calculated would be interesting.
Especially interesting for Slomo footage
cheers
I have an error using deface on Windows 10, caused by onnx, saying that providers are explicitly required since ORT 1.9.
ValueError: This ORT build has ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider'] enabled.
Since ORT 1.9, you are required to explicitly set the providers parameter when instantiating InferenceSession.
For example, onnxruntime.InferenceSession(..., providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider',
'CPUExecutionProvider'], ...)
Judging from the file name, https://github.com/Star-Clouds/CenterFace/blob/master/models/onnx/centerface_bnmerged.onnx is probably a model that is equivalent to the currently included one, with batch normalization ops merged into the adjacent convolution layers - a cheap and straightforward optimization for inference. Replacing the current model with the "bnmerged" version, deface runs a few percent faster than earlier in preliminary tests (CPU execution with opencv and onnxruntime backends).
TODO:
Please look into adding 'OpenVINOExecutionProvider' to the list of EP's in centerface.py, this offers some Intel acceleration that is better than the standard CPU for Intel chips.
Deface is quite handy for processing individual images, but it consistently encounters errors when I attempt to process MP4 files, and I'm uncertain of the underlying cause.
OS: macOS sonoma 14.3
Chip: Apple M1 Pro
Python Version: Python 3.11.7
Traceback (most recent call last):
File "/opt/homebrew/bin/deface", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/deface/deface.py", line 398, in main
video_detect(
File "/opt/homebrew/lib/python3.11/site-packages/deface/deface.py", line 142, in video_detect
nframes = reader.count_frames()
^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/imageio/plugins/ffmpeg.py", line 385, in count_frames
return cf(self._filename)[0]
^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/lib/python3.11/site-packages/imageio_ffmpeg/_io.py", line 187, in count_frames_and_secs
raise RuntimeError("Could not get number of frames") # pragma: no cover
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Could not get number of frames
I don't know if this is the right place to ask. I think deface works very well. It would be great if a GUI and/or mobile apps (for example an Android version) with the deface engine, would be created. So people without technical knowledge or if they only have a mobile phone available, can also benefit from this great tool. Thank you for considering.
Instead of crashing when processing a transparent PNG, I recommend automatically removing the alpha (opacity) channel from the image before processing it.
[ERROR:0] global /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/dnn.cpp (3509) getLayerShapesRecursively OPENCV/DNN: [Convolution]:(363): getMemoryShapes() throws exception. inputs=1 outputs=0/1 blobs=1
[ERROR:0] global /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/dnn.cpp (3515) getLayerShapesRecursively input[0] = [ 1 1 768 1024 ]
[ERROR:0] global /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/dnn.cpp (3523) getLayerShapesRecursively blobs[0] = CV_32FC1 [ 32 3 3 3 ]
[ERROR:0] global /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/dnn.cpp (3525) getLayerShapesRecursively Exception message: OpenCV(4.5.3) /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/layers/convolution_layer.cpp:385: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 1 in function 'getMemoryShapes'
Traceback (most recent call last):
File "/usr/local/bin/deface", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/deface/deface.py", line 312, in main
image_detect(
File "/usr/local/lib/python3.8/dist-packages/deface/deface.py", line 171, in image_detect
dets, _ = centerface(frame, threshold=threshold)
File "/usr/local/lib/python3.8/dist-packages/deface/centerface.py", line 88, in __call__
heatmap, scale, offset, lms = self.net.forward(self.onnx_output_names)
cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-afu9cjzs/opencv/modules/dnn/src/layers/convolution_layer.cpp:385: error: (-2:Unspecified error) Number of input channels should be multiple of 3 but got 1 in function 'getMemoryShapes'
Tried to process mp4 file, but keep getting an error.
AttributeError: 'Reader' object has no attribute 'count_frames'
Here is colab notebook
https://colab.research.google.com/drive/1eptau2-WbM2Ppwxv6C4Zj8RstsucHdKF?usp=sharing
Unfortunately, I can't use the extended processing power of my GPU under Archlinux. I have installed CUDA10 for testing, default is already CUDA11 here. @microsoft can you help with onnxruntime?
% deface --backend onnxrt --scale 1280x720 person1.mp4
Traceback (most recent call last):
File "/home/USer/Videos/Walk/Projekt/videoPython/bin/deface", line 8, in <module>
sys.exit(main())
File "/home/USer/Videos/Walk/Projekt/videoPython/lib/python3.7/site-packages/deface/deface.py", line 277, in main
centerface = CenterFace(in_shape=in_shape, backend=backend)
File "/home/USer/Videos/Walk/Projekt/videoPython/lib/python3.7/site-packages/deface/centerface.py", line 38, in __init__
import onnxruntime
File "/home/USer/Videos/Walk/Projekt/videoPython/lib/python3.7/site-packages/onnxruntime/__init__.py", line 13, in <module>
from onnxruntime.capi._pybind_state import get_all_providers, get_available_providers, get_device, set_seed, \
File "/home/USer/Videos/Walk/Projekt/videoPython/lib/python3.7/site-packages/onnxruntime/capi/_pybind_state.py", line 9, in <module>
import onnxruntime.capi._ld_preload # noqa: F401
File "/home/USer/Videos/Walk/Projekt/videoPython/lib/python3.7/site-packages/onnxruntime/capi/_ld_preload.py", line 12, in <module>
_libcublas = CDLL("libcublas.so.10", mode=RTLD_GLOBAL)
File "/usr/lib/python3.7/ctypes/__init__.py", line 364, in __init__
self._handle = _dlopen(self._name, mode)
OSError: libcublas.so.10: cannot open shared object file: No such file or directory
deface --backend onnxrt --scale 1280x720 person1.mp4 1,37s user 3,96s system 1231% cpu 0,433 total
cycler==0.10.0
decorator==4.4.2
deface==1.0.0
imageio==2.9.0
imageio-ffmpeg==0.4.2
kiwisolver==1.3.1
matplotlib==3.3.3
networkx==2.5
numpy==1.19.4
onnx==1.8.0
onnxruntime-gpu==1.6.0
opencv-python==4.4.0.46
Pillow==8.0.1
protobuf==3.14.0
pyparsing==2.4.7
python-dateutil==2.8.1
PyWavelets==1.1.1
scikit-image==0.18.0
scipy==1.5.4
six==1.15.0
tifffile==2020.12.8
tqdm==4.54.1
typing-extensions==3.7.4.3
Hello,
On a fresh install I have very poor performances, looking into it it seems my CPU is doing all the work and my GPU isn't doing anything despite installing the required pip modules.
Is there a log I can provide you to help debug this issue?
Hi,
I'm trying to find a way to do face bluring on live streams. I tried to use deface with live streamings (through rtsp protocol by example), but it seems it is not implemented.
FFmpeg can natively take a rtsp stream as source, and deface seems to be able to use live feed through the webcam, so I wonder if there is a way to use it with rtsp also ?
Regards,
Romain.
First, thank you for an amazing package!
I'm trying to process an entire directory like this:
deface "pics/" --replacewith mosaic --mosaicsize 20 --thres 0.4 --output "_mosaic20.jpg"
I tried different parameters and would like to use a different postfix for them (e.g. "mosaic20"). The issue is that the output path will be the same for all images (or I'm not understanding something).
IMO, it would make a lot of sense to add a parameter "postfix" that simply adds a specified postfix to all outputs.
--thresh
value is preferable.Solution
--scorethresh
that can be tuned to throw out masks with lower scores.--scalelim
that can be tuned to throw out masks unrealistically large for the scene in consideration.Hello I have run one of my videos through the program over several days to make faces unrecognizable. I have already expected that there will also be false detections. These I wanted to overlay with the original video, because in my opinion it should be the same video only with faces pixelated.
Unfortunately, I noticed today after several days that the source movie and destination movie are no longer the same length.
The word Laufzeit is German for playback length. I am currently trying to adjust the frame rate of the two videos.
source video:
General
Complete name : *.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/mp41)
File size : 152 GiB
Duration : 55 min 15 s
Overall bit rate : 393 Mb/s
Writing application : Lavf58.45.100
Video
ID : 1
Format : HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L5@Main
Codec ID : hev1
Codec ID/Info : High Efficiency Video Coding
Duration : 55 min 15 s
Bit rate : 393 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.015008 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 1.578
Stream size : 152 GiB (100%)
Title : Hisilicon VIDEO
Language : English
Codec configuration box : hvcC
Audio
ID : 2
Format : AAC LC
Format/Info : Advanced Audio Codec Low Complexity
Codec ID : mp4a-40-2
Duration : 55 min 15 s
Bit rate mode : Constant
Bit rate : 125 kb/s
Channel(s) : 1 channel
Channel layout : C
Sampling rate : 48.0 kHz
Frame rate : 46.875 FPS (1024 SPF)
Compression mode : Lossy
Stream size : 49.6 MiB (0%)
Title : Hisilicon AUDIO
Language : English
Default : Yes
Alternate group : 1
the defaced video:
General
Complete name : *_anonymized.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 28.0 GiB
Duration : 55 min 14 s
Overall bit rate : 72.6 Mb/s
Writing application : Lavf58.29.100
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : [email protected]
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 55 min 14 s
Bit rate : 72.6 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 30.020000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.292
Stream size : 28.0 GiB (100%)
Writing library : x264 core 159 r2991 1771b55
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=25.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
Codec configuration box : avcC
Looking at the ffmepg-imageio docs, there does appear to be a parameter called "fps" that changes the output frames per second rate of the anonymized video. However, when setting ffmpeg-config '{"fps": 15} I get an error of: got multiple values for argument 'fps'
How do you properly set that parameter so that the output video is 15 frames per second?
From the results it seems that Deface detects faces in each frame individually.
This causes the blurred regions to jump wildly, sometimes a false positive just blinks in 1 frame, and what's worst:
Sometimes the face slips into 1 or 2 frames, if obscured by even something small. Then the whole point of blurring is gone.
Deface could work in two passes:
This would have benefits:
Sounds good?
Daface works find, but slow. I tried to install onnruntime on apple M1 and I can import onnx and onnexruntime in PyCharm, but these libraries are not available when I run DEFACE in terminal. Is there any trick to either run DEFACE in PyCharm or use onnx properly on m1?
After doing the install on windows (CUDA, cudnn, onnxruntime-gpu), I get the following error when running deface...
CUDA_PATH is set but CUDA wasn't able to be loaded.
I've watched countless videos on setting correct PATHs and so forth and am continuing to have the issue.
Sometimes there's a scene with a fixed number of people.
So there is an assumption they each have 1 face.
deface --facesCount=3
This would reduce the blurred regions to just the 3 top candidates.
By having such parameter, the false positives could be reduced.
Hi!
I'm trying to execute this command:
deface --thresh 0.5 --ffmpeg-config '{"codec": "libx264",
"macro_block_size": 1, "ffmpeg_params": ["-crf","18", "-vf", "vidstabtransform=input=RPRq.trf:smoothing=10,smartblur=lr=1.5:ls=-0.25:lt=-3.5:cr=0.75:cs=0.250:ct=0.5"]}' RPRq-2.mp4
to execute all those three commands at once (tested and works for ffmpeg v5:
deface --thresh 0.5 --ffmpeg-config '{"codec": "libx264",
"macro_block_size": 1, "ffmpeg_params": ["-crf","18"]}' RPRq-2.mp4
ffmpeg -loglevel quiet -stats -i RPRq-2_anonymized.mp4 -vf vidstabtransform=input=RPRq.trf:smoothing=10 -crf 18 RPRq-3.mp4
ffmpeg -loglevel quiet -stats -i RPRq-3.mp4 -vf "smartblur=lr=1.5:ls=-0.25:lt=-3.5:cr=0.75:cs=0.250:ct=0.5" -crf 18 RPRq-4.mp4
But I'm getting this:
Input: RPRq-2.mp4
Output: RPRq-2_anonymized.mp4
0%| | 1/1797 [00:00<24:10, 1.24it/s]Traceback (most recent call last):
File "/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/_io.py", line 630, in write_frames
p.stdin.write(bb)
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/san/.local/bin/deface", line 8, in <module>
sys.exit(main())
File "/home/san/.local/lib/python3.8/site-packages/deface/deface.py", line 360, in main
video_detect(
File "/home/san/.local/lib/python3.8/site-packages/deface/deface.py", line 162, in video_detect
writer.append_data(frame)
File "/home/san/.local/lib/python3.8/site-packages/imageio/core/format.py", line 590, in append_data
return self._append_data(im, total_meta)
File "/home/san/.local/lib/python3.8/site-packages/imageio/plugins/ffmpeg.py", line 604, in _append_data
self._write_gen.send(im)
File "/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/_io.py", line 637, in write_frames
raise IOError(msg)
OSError: [Errno 32] Broken pipe
FFMPEG COMMAND:
/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2 -y -f rawvideo -vcodec rawvideo -s 960x1080 -pix_fmt rgb24 -r 29.97 -i - -an -vcodec libx264 -pix_fmt yuv420p -v warning -vf vidstabtransform=input=RPRq.trf:smoothing=10,smartblur=lr=1.5:ls=-0.25:lt=-3.5:cr=0.75:cs=0.250:ct=0.5 /media/Data/GoPro/test/RPRq-2_anonymized.mp4
FFMPEG STDERR OUTPUT:
0%| | 1/1797 [00:02<1:03:25, 2.12s/it]
There is no problem with rights to write to file and I don't have any other idea...
And verbose output:
deface --thresh 0.5 --ffmpeg-config '{"codec": "libx264", "macro_block_size": 1, "ffmpeg_params": ["-vf", "vidstabtransform=input=RPRq.trf:smoothing=10,smartblur=lr=1.5:ls=-0.25:lt=-3.5:cr=0.75:cs=0.250:ct=0.5", "-v", "40"]}' RPRq-2.mp4
Input: RPRq-2.mp4
Output: RPRq-2_anonymized.mp4
0%| | 1/1797 [00:00<17:38, 1.70it/s]Input #0, rawvideo, from 'pipe:':
Duration: N/A, start: 0.000000, bitrate: 745749 kb/s
Stream #0:0: Video: rawvideo, 1 reference frame (RGB[24] / 0x18424752), rgb24, 960x1080, 745749 kb/s, 29.97 tbr, 29.97 tbn, 29.97 tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
[Parsed_vidstabtransform_0 @ 0x6ddd080] vidstabtransform filter: init v1.1 (2015-05-16)
[Parsed_smartblur_1 @ 0x6de7a00] luma_radius:1.500000 luma_strength:-0.250000 luma_threshold:-4 chroma_radius:0.750000 chroma_strength:0.250000 chroma_threshold:0
[graph 0 input from stream 0:0 @ 0x6dea380] w:960 h:1080 pixfmt:rgb24 tb:100/2997 fr:2997/100 sar:0/1 sws_param:flags=2
[auto_scaler_0 @ 0x6dee280] w:iw h:ih flags:'bicubic' interl:0
[Parsed_smartblur_1 @ 0x6de7a00] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_vidstabtransform_0' and the filter 'Parsed_smartblur_1'
[Parsed_vidstabtransform_0 @ 0x6ddd080] Video transformation/stabilization settings (pass 2/2):
[Parsed_vidstabtransform_0 @ 0x6ddd080] input = RPRq.trf
[Parsed_vidstabtransform_0 @ 0x6ddd080] smoothing = 10
[Parsed_vidstabtransform_0 @ 0x6ddd080] optalgo = gauss
[Parsed_vidstabtransform_0 @ 0x6ddd080] maxshift = -1
[Parsed_vidstabtransform_0 @ 0x6ddd080] maxangle = -1.000000
[Parsed_vidstabtransform_0 @ 0x6ddd080] crop = Keep
[Parsed_vidstabtransform_0 @ 0x6ddd080] relative = True
[Parsed_vidstabtransform_0 @ 0x6ddd080] invert = False
[Parsed_vidstabtransform_0 @ 0x6ddd080] zoom = 0.000000
[Parsed_vidstabtransform_0 @ 0x6ddd080] optzoom = Static (1)
[Parsed_vidstabtransform_0 @ 0x6ddd080] interpol = Bi-Linear (2)
[vidstabtransform @ 0x7ffd5bcfcd50] Final zoom: 9.524674
[auto_scaler_0 @ 0x6dee280] w:960 h:1080 fmt:rgb24 sar:0/1 -> w:960 h:1080 fmt:yuv420p sar:0/1 flags:0x4
Traceback (most recent call last):
File "/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/_io.py", line 630, in write_frames
p.stdin.write(bb)
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/san/.local/bin/deface", line 8, in <module>
sys.exit(main())
File "/home/san/.local/lib/python3.8/site-packages/deface/deface.py", line 360, in main
video_detect(
File "/home/san/.local/lib/python3.8/site-packages/deface/deface.py", line 162, in video_detect
writer.append_data(frame)
File "/home/san/.local/lib/python3.8/site-packages/imageio/core/format.py", line 590, in append_data
return self._append_data(im, total_meta)
File "/home/san/.local/lib/python3.8/site-packages/imageio/plugins/ffmpeg.py", line 604, in _append_data
self._write_gen.send(im)
File "/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/_io.py", line 637, in write_frames
raise IOError(msg)
OSError: [Errno 32] Broken pipe
FFMPEG COMMAND:
/home/san/.local/lib/python3.8/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux64-v4.2.2 -y -f rawvideo -vcodec rawvideo -s 960x1080 -pix_fmt rgb24 -r 29.97 -i - -an -vcodec libx264 -pix_fmt yuv420p -v warning -vf vidstabtransform=input=RPRq.trf:smoothing=10,smartblur=lr=1.5:ls=-0.25:lt=-3.5:cr=0.75:cs=0.250:ct=0.5 -v 40 /media/Data/GoPro/test/RPRq-2_anonymized.mp4
FFMPEG STDERR OUTPUT:
0%| | 1/1797 [00:01<52:42, 1.76s/it]
My preferred way is the "mosaic" option, much more natural than the other ones.
However, it would make a lot of sense to enable the squares to be a percentage of the face size, instead of a number of pixels. If you look at the images attached, it becomes very clear that it's not suited for faces that represent a significant portion of the image.
When using deface cam or deface '' the frames per second are inconsistent. On slower hardware, it might only blur faces at say 11 frames per second, but it will still include meta data saying its 30 fps. Therefore, when played back, it plays too fast. Or with faster hardware (like with a GPU), deface will blur faces at more than 30 fps and yet still include meta data saying its at 30 fps which results in video that looks like its in slow motion.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.