uoa-eresearch / openface_mass_compare Goto Github PK

An openface script that runs a REST server. Posted images are compared against a large dataset, and the most likely match is returned. Works with https://hub.docker.com/r/uoacer/openface-mass-compare/

License: Apache License 2.0

Python 92.41% Shell 3.33% HTML 4.26%

facial recognition openface docker

openface_mass_compare's Introduction

openface_mass_compare

Setup

Create a folder called "images". Create a folder for each person, and place at least one image for each person in their folder
Optionally, add a file called "data.json" containing additional information about each person in the same folder as this script
This folder containing your images directory and data.json should be mounted into the container as /root/data when you run the below docker run command in this directory

Installation

sudo su
apt-get install docker
docker pull uoacer/openface-mass-compare
docker run --name=omc --restart=always --detach=true --volume="`pwd`:/root/data" --publish="8000:8000" --net=host uoacer/openface-mass-compare
The volume command mounts the real directory on the left of the colon to the /root/data directory in the container
The first time you run this, it'll create a 2D matrix, where each element is an image from the images folder, processed by the Torch7 network
Once it's done, it'll save the result to data.pickle, for faster startup next time. For a dataset of 3678 images building this pickle file takes ~6.8 minutes
To view the logs, run the command
docker logs -f omc
To test, run
time curl localhost:8000 --data-binary @image.jpg -vv

Credits

openface_mass_compare's People

Contributors

Stargazers

Watchers

Forkers

itmgr nomo278 twobob wyc2015fq thewewokachronicle spring3th modster rahimnathwani

openface_mass_compare's Issues

Live Refreshing

Good Morning,
Is it possible to reload the images / data.Jason in real-time?? It would be really better.
Thank you,
Gabriel

multiprocessing: Number of processes must be at least 1

The container keeps crashing with this error:

generating pickle
Traceback (most recent call last):
  File "generate_pickle.py", line 27, in <module>
    p = Pool(processes=PROCESSES, initializer=init)
  File "/usr/lib/python2.7/multiprocessing/__init__.py", line 232, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild)
  File "/usr/lib/python2.7/multiprocessing/pool.py", line 152, in __init__
    raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1
[2017-02-01 10:14:49 +0000] [9] [INFO] Starting gunicorn 19.6.0
[2017-02-01 10:14:49 +0000] [9] [INFO] Listening at: http://0.0.0.0:8000 (9)
[2017-02-01 10:14:49 +0000] [9] [INFO] Using worker: sync
[2017-02-01 10:14:49 +0000] [14] [INFO] Booting worker with pid: 14
[2017-02-01 10:14:49 +0000] [16] [INFO] Booting worker with pid: 16
[2017-02-01 10:14:49 +0000] [18] [INFO] Booting worker with pid: 18
[2017-02-01 10:14:49 +0000] [20] [INFO] Booting worker with pid: 20
[2017-02-01 10:14:49 +0000] [22] [INFO] Booting worker with pid: 22
[2017-02-01 10:14:49 +0000] [24] [INFO] Booting worker with pid: 24
[2017-02-01 10:14:49 +0000] [26] [INFO] Booting worker with pid: 26
[2017-02-01 10:14:49 +0000] [28] [INFO] Booting worker with pid: 28
[2017-02-01 10:14:57 +0000] [14] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 557, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 136, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 357, in import_app
    __import__(module)
  File "/root/web_server.py", line 32, in <module>
    with open("/root/data/data.pickle") as f:
IOError: [Errno 2] No such file or directory: '/root/data/data.pickle'
[2017-02-01 10:14:57 +0000] [14] [INFO] Worker exiting (pid: 14)
[2017-02-01 10:14:58 +0000] [16] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 557, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 136, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 357, in import_app
    __import__(module)
  File "/root/web_server.py", line 32, in <module>
    with open("/root/data/data.pickle") as f:
IOError: [Errno 2] No such file or directory: '/root/data/data.pickle'
[2017-02-01 10:14:58 +0000] [16] [INFO] Worker exiting (pid: 16)
Traceback (most recent call last):
  File "/usr/local/bin/gunicorn", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 74, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]").run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 192, in run
    super(Application, self).run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 72, in run
    Arbiter(self).run()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 218, in run
    self.halt(reason=inst.reason, exit_status=inst.exit_status)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 331, in halt
    self.stop()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 381, in stop
    time.sleep(0.1)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 231, in handle_chld
    self.reap_workers()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 506, in reap_workers
    raise HaltServer(reason, self.WORKER_BOOT_ERROR)
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>

Container info:

[
    {
        "Id": "b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e",
        "Created": "2017-02-01T09:38:24.86630531Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "/root/run.sh"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 1,
            "Error": "",
            "StartedAt": "2017-02-01T10:15:09.720998646Z",
            "FinishedAt": "2017-02-01T10:15:20.209644949Z"
        },
        "Image": "sha256:45f15a4cbfa13c0b1086579b54b4a408651c95d9ddc6955627dcb248c0486d4b",
        "ResolvConfPath": "/var/lib/docker/containers/b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e/hostname",
        "HostsPath": "/var/lib/docker/containers/b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e/hosts",
        "LogPath": "/var/lib/docker/containers/b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e/b11964bb9e343d22d239912ff7924d043a856cc326b99c0bec8bd82d1a4c5c2e-json.log",
        "Name": "/omc",
        "RestartCount": 2,
        "Driver": "aufs",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [
                "/home/dev/openface:/root/data"
            ],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {}
            },
            "NetworkMode": "host",
            "PortBindings": {
                "8000/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "8000"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "always",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": null,
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": -1,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Name": "aufs",
            "Data": null
        },
        "Mounts": [
            {
                "Type": "bind",
                "Source": "/home/dev/openface",
                "Destination": "/root/data",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "dev",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8000/tcp": {},
                "9000/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "/root/run.sh"
            ],
            "ArgsEscaped": true,
            "Image": "uoacer/openface-mass-compare",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "1442563112f0300eaa0b03f9fc5c5fab07e44255bca1c6894c3d4409beaff7e9",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": null,
            "SandboxKey": "/var/run/docker/netns/default",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "host": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "dc5c3184582184cdfdd6cf3297e6fb5e781b2f36005488d54960bb2f88e4e39a",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": ""
                }
            }
        }
    }
]

where to put images folder, localhost, etc?

Hey, I used docker with Digital Command, and put the images folder (and data.json) in the server there.
About time curl localhost:8000 --data-binary @image.jpg -vv, is the image.jpg supposed to be one of the images of one of the person? Right now, if I try to do that, it just says "Couldn't read data from "nameOfImage.jpg", this makes an empty POST"

Also, do I need to change the OpenFaceUrl from "http://ml.cer.auckland.ac.nz:8000" to something else as well?
Thanks in advance

Proccess lua of multiprocessing not stop

Hi;
You know becase process lua not stop after the generate_pickle.py finalize?
Thanks by your shared code.

How might one rework getRep to return mulitple instances.

def getRep(bgrImg, align=align, net=net):
    rgbImg = cv2.cvtColor(bgrImg, cv2.COLOR_BGR2RGB)
    bb = align.getLargestFaceBoundingBox(rgbImg)
    if bb is None:
        raise Exception("Unable to find a face")
    alignedFace = align.align(96, rgbImg, bb, landmarkIndices=openface.AlignDlib.OUTER_EYES_AND_NOSE)
    if alignedFace is None:
        raise Exception("Unable to align image")
    rep = net.forward(alignedFace)
    return rep

I tried

def getReps(bgrImg, align=align, net=net):
    holder=[]
    rgbImg = cv2.cvtColor(bgrImg, cv2.COLOR_BGR2RGB)
    bb = align.getAllFaceBoundingBoxes(rgbImg)
    if bb is None:
        raise Exception("Unable to find a face")
    for face in bb:
        alignedFace = align.align(96, rgbImg, face, landmarkIndices=openface.AlignDlib.OUTER_EYES_AND_NOSE)
        if alignedFace is None:
            print("Unable to align image")
            continue
        if not alignedFace is None:
            rep = net.forward(alignedFace)
            print("did rep", rep)
            holder.append(rep)
    return holder

But I am a little confused how to handle to output.
I read http://openface-api.readthedocs.io/en/latest/openface.html
and thought I would jut receive a list of rectangles from align
which I could walk but now I am a little confused as to a) how to check which actual images matched which face and which names that pertained to.

Anyway, it's just a question really. Many thanks

TypeError: cannot concatenate 'str' and 'exceptions.IOError' objects

After running the new image, I get the exceptions below, resulting in a stopped container.
I'm running on Windows with Docker 1.12 Quickstart Terminal, maybe the mounting is a problem?

2017-02-01T21:49:12.672950336Z [2017-02-01 21:49:12 +0000] [24] [ERROR] Exception in worker process
2017-02-01T21:49:12.672961570Z Traceback (most recent call last):
2017-02-01T21:49:12.672964723Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 557, in spawn_worker
2017-02-01T21:49:12.672967468Z     worker.init_process()
2017-02-01T21:49:12.672969895Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
2017-02-01T21:49:12.672977429Z     self.load_wsgi()
2017-02-01T21:49:12.672979807Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 136, in load_wsgi
2017-02-01T21:49:12.672982091Z     self.wsgi = self.app.wsgi()
2017-02-01T21:49:12.672984242Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
2017-02-01T21:49:12.672986423Z     self.callable = self.load()
2017-02-01T21:49:12.672988468Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
2017-02-01T21:49:12.672990588Z     return self.load_wsgiapp()
2017-02-01T21:49:12.672992487Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
2017-02-01T21:49:12.672994724Z     return util.import_app(self.app_uri)
2017-02-01T21:49:12.672996763Z   File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 357, in import_app
2017-02-01T21:49:12.672998886Z     __import__(module)
2017-02-01T21:49:12.673000850Z   File "/root/web_server.py", line 49, in <module>
2017-02-01T21:49:12.673003191Z     print("Unable to load data.json: " + e)
2017-02-01T21:49:12.673005300Z TypeError: cannot concatenate 'str' and 'exceptions.IOError' objects
2017-02-01T21:49:12.673157545Z [2017-02-01 21:49:12 +0000] [24] [INFO] Worker exiting (pid: 24)

No such file or directory: '/root/data/data.pickle'

I get this error when booting the docker. Seems it's unable to find data.pickle and is unable to create a new one?

[2017-09-14 08:35:40 +0000] [27] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/arbiter.py", line 578, in spawn_worker
    worker.init_process()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 126, in init_process
    self.load_wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/workers/base.py", line 135, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 65, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python2.7/dist-packages/gunicorn/util.py", line 352, in import_app
    __import__(module)
  File "/root/web_server.py", line 24, in <module>
    from util import *
  File "/root/util.py", line 23, in <module>
    with open("/root/data/data.pickle") as f:
IOError: [Errno 2] No such file or directory: '/root/data/data.pickle'

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.