johanmodin / clifs Goto Github PK
View Code? Open in Web Editor NEWContrastive Language-Image Forensic Search allows free text searching through videos using OpenAI's machine learning model CLIP
License: Apache License 2.0
Contrastive Language-Image Forensic Search allows free text searching through videos using OpenAI's machine learning model CLIP
License: Apache License 2.0
Great/horrifying research.
Are there any efforts/institutions dedicated to fighting widespread mis/use of surveillance?
Thanks
Hi, @johanmodin Thank you for your great work! But I have faced some problem with clifs:
As shown below, I have created my ckifs_search-engine
, That means that I have created my docker image, right? But I encountered errors while bringing up the project.
Step 15/18 : RUN python3 -c "import clip; clip.load('$MODEL', device='cpu')"
---> Running in 7145f0df5bd7
100%|#######################################| 402M/402M [03:58<00:00, 1.77MiB/s]
Removing intermediate container 7145f0df5bd7
---> a62989bd9503
Step 16/18 : RUN ln -sf /usr/bin/python3 /usr/bin/python
---> Running in a0e80becb3cf
Removing intermediate container a0e80becb3cf
---> fd581f18de7d
Step 17/18 : COPY app/ /app
---> 3525ae426cda
Step 18/18 : CMD python clifs.py
---> Running in 89927927e5ef
Removing intermediate container 89927927e5ef
---> fab0dbbe3a71
Successfully built fab0dbbe3a71
Successfully tagged clifs_search-engine:latest
Creating network "clifs_default" with the default driver
Creating clifs_web_1 ... error
Creating search-engine ...
ERROR: for clifs_web_1 Cannot start service web: OCI runtime create failed: container_linux.go:345:
Creating search-engine ... done
ERROR: for web Cannot start service web: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: "/clifs/entrypoint.sh": permission denied": unknown
ERROR: Encountered errors while bringing up the project.
So I want to know how to fix it and how to use it.
Thank you for your answering!!!
Hi,
I tried to run docker-compose build && docker-compose up
but got the following error:
search-engine | 2021-10-04 05:56:22,647 - Adding video: /app/data/input/sherbrooke_video.avi
search-engine | [ERROR:0] global /tmp/pip-req-build-l1r0y34w/opencv/modules/videoio/src/cap.cpp (160) open VIDEOIO(CV_IMAGES): raised OpenCV exception:
search-engine |
search-engine | OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): /app/data/input/sherbrooke_video.avi in function 'icvExtractPattern'
search-engine |
search-engine |
search-engine | Traceback (most recent call last):
search-engine | File "server.py", line 8, in <module>
search-engine | clifs = CLIFS()
search-engine | File "/app/clifs.py", line 30, in __init__
search-engine | self.add_video(f)
search-engine | File "/app/clifs.py", line 71, in add_video
search-engine | feature_t = torch.cat(feature_list, dim=0)
search-engine | RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CPU, CUDA, QuantizedCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].
Have you faced this issue before ? Do you have an idea on how to solve it ?
nice job,i wonder how can i further train my own data to adaptive more efficiency to my video clips
when running the docker-compose build && docker-compose up
I get an OCI runtime creage failed ( posted below):
Sending build context to Docker daemon 1.724MB
Step 1/7 : FROM python:3
---> 6beb0d435def
Step 2/7 : ENV PYTHONUNBUFFERED=1
---> Using cache
---> 2aab34bae599
Step 3/7 : WORKDIR /clifs
---> Using cache
---> 7c6854350226
Step 4/7 : COPY requirements.txt /clifs
---> Using cache
---> df31f0e3f4ff
Step 5/7 : RUN pip install -r requirements.txt
---> Running in 8eca7fd2853d
OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:722: waiting for init preliminary setup caused: EOF: unknown
ERROR: Service 'web' failed to build : Build failed ```
**any ideas I have tried running the command as sudo and user. Upon my digging I found the error might contain to setup of Dockerfile but not sure.**
when running the docker-compose build && docker-compose up I get an error ( posted below):
Step 1/7 : FROM python:3
Sending build context to Docker daemon 4.854kB
Step 1/14 : FROM nvidia/cuda:11.2.2-base-ubuntu20.04 AS clifs
2 errors occurred:
* Status: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), Code: 1
* Status: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers), Code: 1
Any Suggestion?
Would love to run this, but had some trouble getting the GPU version running as it doesn't support my GPU. I tried using a newer version of the pytorch base image to get past this, but ran into some issues.
hi, i meet a question about this project. When i use the command 'docker-compose build && docker-compose up', if i use the original codes of docker-compose.yml, the output seems wrong, like this:
And if i change the code of 'command' in the docker-compose.yml to 'sh /clifs/entrypoint.sh', the output is this:
they can not fins the entrypoint.sh, Please tell me how to fix it, Thank you!
Hey Johan @johanmodin ,
Great work with CLIFS! I am able to set it up as shown in the readme and search from localhost (127.0.0.1). How should I go about if I want to host this with a public URL (I will of course cite and reference your work.)
L16 in entrypoint.sh
does say 0.0.0.0 but somehow it does not serve the hosts public url (I am using EC2).
thanks again and looking forward to hearing from you.
Thanks for the great job!
I think it's a local bug, but for every query the match score seems to be too low.
yellow ambulance - 34.84375
car - 36.625
ambulance - 35.21875
any idea what might be wrong?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.