biometrics / openbr Goto Github PK

I'm not entirely certain this is the best place for this comment, but it was the best I found.

When first trying to install openbr, from source on an Ubuntu 13.04 system with an Nvidia graphics card, I ran into a problem when installing OpenCV. Apparently, CUDA does not support (as of 6/13/13) gcc4.7. So, before running the cmake cmd, I used update-alternatives to temporarily let gcc4.6 be the compiler version used to build Opencv.

Hope this helps ......

Hey guys (especially jklontz),

I've deployed openbr to a new type of hardware and have run into (what looks like) processor constraints. I'd like to get your expert insight and see if this sounds funny to you.

First, I'm using the same 0.3.0 openbr build from a month or so ago. I know you'd recommend getting the latest, but this version worked for me during my 2-week pilot and I'm trying to standardize everything for ease of mass deployment. I run it in "br -daemon /pathto/brpipe" mode and then "echo" commands to it.

My first hardware configuration was a Lenovo Thinkpad laptop: Intel core i5 4 cores @ 2.4 Ghz. My comparison speeds were (are -- I still have the hardware here) around 1.5e+04. The processing of a single image against the known gallery seemed almost instantaneous. The "br" process in top maxed out around 40% cpu utilization as one image per second was fed in.

My second hardware config is a booksize miniPC with a Dual core Atom D525 @ 1.8 Ghz. My br comparison speeds are reading as 3.1e+03. Images seem to take more than a second to process and the "br" process in top stays at 100% (or very close).

Everything but the hardware is the same. Same OpenCV version. Same Qt 5.0.1. Same linux distro. Neither machine has any heavy processes beforehand. Both machines have SSD drives.

*** Late addition ***

I just noticed something that probably holds the key to the issue: On the Atom, the first time I send an image through to the br daemon, it is slow, just as described above... a second or two to process. And, naturally, since my application is constantly producing new images, they all seem slow to process.

HOWEVER, if I send the same image through a second time (or any number of subsequent times), it's almost instantaneous, just as it should be.

What does this mean? File read/write issues (both have SSDs)? Memory 6 GB vs 2 GB? Processor cache?

To me, it screams disk I/O issue. The first process has to read from disk. The second from memory. But how can that be when the Atom PC has a SSD just like the laptop? Is it possible the CPU is too slow to utilize the SSD's full I/O speed?

Atom pc page: http://www.zotacusa.com/zbox-sd-id12.html
Laptop specs: http://support.lenovo.com/en_US/product-and-parts/detail.page?DocID=PD015731

(Sorry this has turned into a computing general knowledge question. I thought it was br-specific when I started writing.)

I'm finding that my passively running "br -deamon < pipe" uses 100% of one of the cores in my quad core processor. After a while, my laptop (where it's running) feels a bit warm. It's mostly sleeping, though, so shouldn't it be pretty calm on the CPU usage?

Hi,

I'm using latest version of openbr for Windows. In case that output file (scores2.csv) is locked for writing (it's opened by other application), the br.exe will trow following exception but it will not close it execution (the br will be still running and can't be closed without closing whole command window).

Is it possible to terminate the execution of br.exe if this kind of exception will occur?

c:\Program Files\OpenBR\bin>br.exe -algorithm AgeEstimation -enroll "c:\p1.jpg"
"c:\scores2.csv"
Set algorithm to AgeEstimation
Loading AgeRegression
Enrolling c:\p1.jpg to c:\scores2.csv
Fatal: Failed to open c:\scores2.csv for writing.
SDK Path: C:/Program Files/OpenBR
File: C:\openbr\openbr\core\qtutils.cpp
Function: void __cdecl QtUtils::writeFile(const class QString &,const class QB
yteArray &,int)
Line: 122

This could just be a matter of updating the example calls or maybe something more significant, or something that's getting misconfigured using the install instructions from here

WA-003458-AP:openbr joshua.noble$ br -algorithm FaceRecognition \
>      -compare data/MEDS/img/S354-01-t10_01.jpg data/MEDS/img/S354-02-t10_01.jpg \
>      -compare data/MEDS/img/S354-01-t10_01.jpg data/MEDS/img/S386-04-t10_01.jpg
Set algorithm to FaceRecognition
Comparing data/MEDS/img/S354-01-t10_01.jpg and data/MEDS/img/S354-02-t10_01.jpg
OpenCV Error: Assertion failed (map1.size().area() > 0) in remap, file /Users/joshua.noble/Downloads/opencv-2.4.5/modules/imgproc/src/imgwarp.cpp, line 2940
Warning: Exception triggered when processing data/MEDS/img/S354-01-t10_01.jpg[Affine_0=(223,242), Affine_1=(314,244), DFFS=0, FTE=false, FTO=false, Face=(147,145,250,250), First_Eye=(223,242), Gallery=data/MEDS/img/S354-01-t10_01.jpg, Index=0, Points=[(223,242), (314,244)], Rects=[(147,145,250,250)], Second_Eye=(314,244)] with transform RndSubspace
  File: /Users/joshua.noble/code/openbr/openbr/openbr_plugin.cpp
  Function: void _project(const br::Transform *, const br::Template *, br::Template *)
  Line: 1144
Fatal: Empty template.
  File: /Users/joshua.noble/code/openbr/./openbr/openbr_plugin.h
  Function: const cv::Mat &br::Template::m() const
  Line: 330
Abort trap: 6

Currently, openbr doesn't provide the functionality in br::enroll to add templates to an already existing gallery. This would be a timesaving feature for large galleries.

If, for example, the limit for ranked retrieval is 200 but filtering using demographics removes all but 3 images in the gallery, the results should only include those 3 images and not the 3 images + first 197 images enrolled in gallery.

Please support android

I have an algorithm that includes the following as one of a list of transforms (argument to Stream in this case, although that doesn't matter):
(Cvt(Gray)+PP5Register+Affine(60, 60, .25, .2167)+AgeBIF+ExternalFilter++Discard)/Identity+Draw+DrawPropertyPoint(Affine_0,Label)

File::parse attempts to interpret this string as a point, calling QtUtils::toPoint which drops the start/end parentheses and calls QtUtils::parse on the following:
Cvt(Gray)+PP5Register+Affine(60, 60, .25, .2167)+AgeBIF+ExternalFilter++Discard)/Identity+Draw+DrawPropertyPoint(Affine_0,Label

This results in a qFatal because of unbalanced parenthesis for DrawPropertyPoint, of course the ) corresponding to the initial ( is after discard, not at the end of the string so in that sense QtUtils::toPoint removed the wrong ). This can be considered a simple bug, since the expected behavior here is for QtUtils::toPoint to fail, although I still think problems could be avoided by using information about the expected type of arguments (in algorithm strings), rather than attempting to infer the type from the string.

Here's the command I'm using:

br -algorithm "Open+Cascade(FrontalFace)+ASEFEyes+Draw+Show" -enrollAll -enroll imagesdirectory

The images can be gotten here:

https://s3.amazonaws.com/cyrus_random_images/kari_hall.bz2

Linux: 3.6.10-4.fc18.x86_64
gcc-c++-4.7.2-8.fc18.x86_64
libgcc-4.7.2-8.fc18.x86_64
gcc-4.7.2-8.fc18.x86_64

It segfaults out on the 3rd or 4th image can't tell exactly what's causing it.

Trying to duplicate what was in evalFaceRecognition.sh, specifically:

br -algorithm='FaceRecognition' ...

I used instead:

br -algorithm='FaceDetection!<FaceRecognitionRegistration>!<FaceRecognitionExtraction>+<FaceRecognitionEmbedding>+<FaceRecognitionQuantization>:MatchProbability(ByteL1)' ...

which is what is specified in plugins/algorithms.cpp.

However - that did not work:

 Warning: Encountered 361564 NaN scores!

and

 Fatal: No genuine scores!

Apparently, the -compare generates a .mtx file with null distances?

See https://groups.google.com/forum/#!topic/openbr-dev/oWtf1k0-3DU

Need to change the hosted source such that the models are included. This will facilitate the HomeBrew formula, among other things.

If you enroll a large number of files (e.g. using a wildcard within a folder), the name for the gallery will be every file you enrolled. This makes template size potentially massive. Right now, a simple workaround is to use the Retain transform to only keep certain metadata (not Gallery) but that shouldn't be necessary.

Hey guys,
Thanks for the great work. This library would really help me with my occasional hobby projects. I already have some cool stuff in my head which I will start churning out once it this puzzling issue is solved.

Issue :

So, I have a Ubuntu 13.04[1] machine and my every time I compile openbr, it compiles fine but while executing it crashes with Segmentation fault[4].

How To reproduce :

Well executing br , alone did it :)

On further investigation I found out that although openbr cmake was loading header files from qt5 it was linking to qt4 libraries [2][3][5][6].

I have been literally hitting my head on this issue for past 2 days, any help would be pretty cool.

References :

[1] OS Version

:~/code/openbr/build$ lsb_release -a
Distributor ID: Ubuntu
Description: Ubuntu 13.04

[2] Qt5 Version

I have compiled my own Qt5
~/code/openbr/build$ qmake --version
QMake version 3.0
Using Qt version 5.0.2 in /usr/lib/i386-linux-gnu

[3] Qt4 Version

This is stock qt4 that comes with Ubuntu
~/code/openbr/app$ qmake-qt4 --version
QMake version 2.01a
Using Qt version 4.8.4 in /usr/lib/i386-linux-gnu

[4] Crash dump

(gdb) bt
~/code/openbr/app/br$ gdb br
0. 0xb369f5e2 in ?? () from /usr/lib/i386-linux-gnu/ libQtTest.so.4

1. 0xb7fed216 in ?? () from /lib/ld-linux.so.2
2. 0xb7fed2fc in ?? () from /lib/ld-linux.so.2
3. 0xb7fdf1df in ?? () from /lib/ld-linux.so.2

[5] Cmake setup

[6] Crash GDB bt

To reproduce, run this:

Stream([Cvt(Gray)/Cvt(Gray)])

I added some debugs to CompositeTransform::init() to highlight the strangeness:

"Cvt" is not trainable 
"Cvt" is not trainable 
and thus "Fork" is not trainable 
"Fork" is not trainable 
and thus "Stream" is not trainable 
1 training files
Training Enrollment
Warning: How did this happen? You're training a nontrainable transform.

I think the title says it all. With BR_EMBEDDED flag, http access is unavailable at the moment.

Thanks, guys. Especially jklontz. Huge help on #openbr today.

I built against Qt 5.1 on windows, there were no compile issues, however install failed since InstallDependencies.cmake looks for icuin49.dll, icuuc49.dll, and icudt49.dll--but Qt has switched to a new version of these libraries and now ships icuin51.dll, icuuc51, and icudt51.dll.

I'm getting a seg fault with the command:

$ br -algorithm "Open+Cascade(FrontalFace)+ASEFEyes+Draw+Show" -enrollAll -enroll facedirectory

Segmentation fault (core dumped)T=94

It goes through images putting a bright green box and eye markers on full color faces. Thought it might be a particular image causing the seg fault, but I removed what I thought was the offending image (and the one before) and it keeps happening at the same point.

@caotto Just discovered it looks like your pull request broke FaceRecognition (commits from yesterday still work). In particular, it looks like @sklum's correct fix in 0572173 broke the compatibility. Template comparison is resulting in NaN scores which is a classic symptom of this error. We can investigate further tomorrow, but the fix is almost certainly to re-train again. I don't think the complete retrain of etc. is needed.

I tried face_recognition_search, but sounds like Transform cannot enroll all TemplateList objects. e.g., there are 10 images in the folder and they can be found in br::TemplateList::fromGallery but there would be 6 items (i.e., the "target" after transforming) after:

br::Globals->enrollAll = true; // Enroll 0 or more faces per image
target >> *transform;

The workaround would have to enroll one at a time.

@sklum I hope it's ok that I'm pawning this off on you.

http://www.btas2013.org/call-for-demos/

Due date is 22nd. Let me know if you're too busy though, and I'll take care of it.

Calling clone on a transform that was created with an argument of type QStringList doesn't work correctly.
The description used to rebuild the transform is got by a call to file.flat(), which seems to work well enough, but when the transform is initialized, the stringlist argument ends up in void File::set(const QString &key, const QString &value), which tries to interpret the list as a list of points, or rects, then ultimately decides to interpret it as a list of floats (which does not work).

Do we have a discussion alias for users and developers? Thus it can help our communication. Now sounds like we have to look at the examples and docs by ourselves if encountering an issue. In addition, we have to read code to figure out what is going on.

Using the PP5 plugin for face detection doesn't work for a couple reasons. Even if enrollAll is set, the plugin is still initialized to only detect the best face.

More significantly, even if that is corrected, PP5Enroll will still not store multiple detected faces in the file metadata correctly, because everything detected is stored with named properties, and even if multiple faces are indeed detected, only 1 copy of each property will be kept.

By this point it would actually be nice to change the way the plugin works to output multiple templates, rather than storing multiple matrices in 1 template (and attempting to store multiple detections in the metadata) although I'm not sure whether that would damage any current use cases.

@jklontz Is there still a function for sort a vector and getting the sorted values along with their corresponding index from the original vector? I remember we had something similar in common.cpp before. Couldn't find it now.

Qt 5.0 is out of beta.

To illustrate how GUI interfaces can be built on top of OpenBR.

We need a mechanism to return templates asynchronously for online algorithms (https://groups.google.com/d/topic/openbr-dev/vzGd-b5sw-0/discussion).

@caotto @sklum and I decided on a way to do this over IRC (http://pastebin.com/eWcwAHqj).

@bklare This is the last technical hurdle needed to implement your request as a Transform instead of a Distance. It's not high on my priority list, unless you'd like it to be :)

Pre-built binary release for #102.

In general, there should be many more tutorials but since most people want to use OpenBR in applications this one should be prioritized.

I'd like to propose the following changes be made concurrently to the existing GUI code in the repository:

1. Deprecate Forensic Face. @mburge has suggested for a while now that in the long run we really need a web client frontend for matching instead of the current thick client. I've slowly come to terms with this reality and I think HTML5 is powerful enough that it can now be reasonably implemented. Thus the thick client will be replaced with something along the lines of: br -algorithm FaceRecognition -web <pre-enrolled-target-gallery> to launch br as a web server running listening on port 80/8080/etc serving the thin client application and handling search queries. Note, we can still support the traditional single workstation model by starting the web server and then launching a browser pointed at localhost.
2. Remove qwt as a third party library. This code only exists to make Forensic Face compile and isn't used enough to justify holding on to it. Removing it will dramatically reduce compilation time.
3. Move the existing GUI widget code into the main SDK. This code is still used by @sklum and has value to it. @caotto recent work has made it possible to build GUI components into the algorithm grammar, a potentially very powerful improvement, and this will make it possible to leverage existing GUI code to build new Transforms. In short, the functionality exposed by libopenbr-gui will be rolled into libopenbr.

Transform arguments that are lists (e.g. QList<Transform *>, QList<Distance *> QStringList) etc. are not parsed correctly. Due to a some related changes (use of file to parse algorithm strings, changes to list argument parsing in File::set), explicit list arguments e.g. Pipe([Open,Show]) are not parsed correctly.

In the pipe example, the transform list argument is turned into a property, with the QVariant having type QList, and the QVariants in the list are actually strings. Later, in Object::init, the argument property is retrieved, and its value (which has type QList) cannot be converted directly into a string, so after the actual property name is looked up, its value gets set directly to the QList, which is completely incorrect, since the pipe instance will interpret it as a list of transforms. This is quite bad, and the only reason pipes work at all currently is that they are typically set up using '+', and after that is expanded, the algorithm string isn't converted into a File.

It would be useful to be able to analyze face and landmark detection results in OpenBR. At the least, painting bounding boxes around detected regions could help for qualitatively comparing different detection methods and parameters. More ambitiously, it would be good to provide quantitative results regarding the accuracy of detection (based on ground truth information provided).

Perhaps some of this functionality already exists. If not, then I would be interested in developing such functionality (unless someone else wants to).

@jklontz - Do you have any issues with such tools being built into OpenBR? Detection and alignment are the currently the biggest bottleneck in terms of accuracy for this system. Without such tools it will be difficult to improve these stages.

Trying to compile on Ubuntu (12.04 64 bit), I'm getting a strange error

Building CXX object app/examples/CMakeFiles/age_estimation.dir/age_estimation.cpp.o
In file included from /usr/include/qt5/QtCore/qnamespace.h:45:0,
                 from /usr/include/qt5/QtCore/qobjectdefs.h:45,
                 from /usr/include/qt5/QtCore/qobject.h:47,
                 from /usr/include/qt5/QtCore/qcoreapplication.h:45,
                 from /usr/include/qt5/QtCore/QCoreApplication:1,
                 from /home/edent/git/openbr/app/br/br.cpp:17:
/usr/include/qt5/QtCore/qglobal.h:1079:4: error: #error "You must build your code with position independent code if Qt was built with -reduce-relocations. " "Compile your code with -fPIC or -fPIE."

I'm following the instructions at http://openbiometrics.org/doxygen/latest/linux_all.html and have all the correct dependencies etc installed.

Am I missing something obvious?

Should improve accuracy and give a 0-1 match score range. Also update examples accordingly.

Similar to the loadMtx file, a saveMtx file needs to be added to allow Matlab users to save similarity matrices for analysis in openbr.

Figure out what changes should be made based on the following feedback:

1. The FAR axis labels were truncated.
2. I want to be able to edit the legend so that it is effortlessly understandable by a client. For example, I covered up openBR’s default legend in Powerpoint and created my own legend in the image below.
3. I sometimes like to order the legend by top performer
4. It is also helpful for Powerpoints to be able to adjust font size of axis labels.
5. I want the ability to have the x,y coordinates and use Excel in case I want to plot different experiments on same graph or remove certain curves
6. I try to keep the same color for a cohort.

1. seems doable.
2. it seems you mean mapping a file name to a name with more textual meaning. This may make more sense to do my actually changing the file name (assuming white space will be preserved).
3. probably difficult given ambiguity of "top performer". For example, ROC curve can switch at different points. Also, a it may not always be a performer (e.g., in glasses study we measure cohorts). If color switches between plots this can be confusing.
4. seems doable (something I would find beneficial as well)
5. is already possibly (will forward a previous email stating how to do this).
6. already exists

@jklontz - how much time do you estimate it would take you to make a local ternary pattern plugin for openbr? Would be interesting to compare it to LBP. Some researchers prefer LTP over LBP.

Tan, Xiaoyang, and Bill Triggs. "Enhanced local texture feature sets for face recognition under difficult lighting conditions." Analysis and Modeling of Faces and Gestures (2007): 168-182.

The share/openbr/models directory in the repository leads nowhere.

Realized this from trying to install openbr on my linux machine.

git submodule init
Submodule 'data' () registered for path 'data'
Submodule 'share/openbr/doc' () registered for path 'share/openbr/doc'
Submodule 'share/openbr/models' () registered for path 'share/openbr/models'

git submodule update
fatal: https://github.com/biometrics/openbr-models.git/info/refs not found: did you run git update-server-info on the server?
Unable to fetch in submodule path 'share/openbr/models'

@jklontz @caotto I am interested in using a COTS plugin to match face images in a modality where face and eye detection generally fail. As such, I want to provide ground truth eye locations in the sigsets, and use these values at the enrollment stage for the COTS matcher.

From examining the COTS plugin code, this operation does not seem to be supported in openbr. Further, it would seem the fix would be to create a new plugin for each corresponding COTS matcher, which uses the meta data provided in the sigset (i.e., eye locations) to enroll the face image. Does these two statements seem correct? If so, then I will likely create at least one such plugin.

I would like to point out that identifiers like "__OPENBR_H" and "__BEE_H" do not fit to the expected naming convention of the C++ language standard.
Would you like to adjust your selection for unique names?

After downloading the sample data, I tried running the example for Face Recognition Evaluation from the command line using the MEDS sample data.

'''
$ br -algorithm FaceRecognition -path ../data/MEDS/img/
-enroll ../data/MEDS/sigset/MEDS_frontal_target.xml target.gal
-enroll ../data/MEDS/sigset/MEDS_frontal_query.xml query.gal
-compare target.gal query.gal scores.mtx
-makeMask ../data/MEDS/sigset/MEDS_frontal_target.xml ../data/MEDS/sigset/MEDS_frontal_query.xml MEDS.mask
-eval scores.mtx MEDS.mask Algorithm_Dataset/FaceRecognition_MEDS.csv
-eval ../data/MEDS/simmat/COTS_MEDS.mtx MEDS.mask Algorithm_Dataset/COTS_MEDS.csv
-plot Algorithm_Dataset/FaceRecognition_MEDS.csv Algorithm_Dataset/COTS_MEDS.csv MEDS
'''

It's looking for the data/MEDS/simmat/COTS_MEDS.mtx file for evaluation, but the simmat folder isn't there. I've tried looking in the MEDS dataset and I can't find the COTS_MEDS.mtx file nor can I find the simmat folder. Is there something I'm missing?

The pre-built version of opencv for windows has a directory structure like:
opencv/build
where build contains several copies of the opencv libraries, broken down by architecture/compiler and a OpenCVConfig.cmake file which picks up the correct pre-compiled version for the generator you are using.

With openbr, adding opencv/build to CMAKE_PREFIX_PATH will result in a working build (since OpenCVConfig.cmake will point to the right library directory); however, install (InstallDependencies.cmake) looks for OpenCV_DIR/bin, which for the pre-built windows distribution is opencv/build/bin, a non-existant directory.

So, can we use a path relative to the opencv lib dir? Something like ${OpenCV_LIB_DIR}/../bin, I think this would work correctly for the pre-built distribution, and also for versions people built themselves.

Of course I tried to do this myself and it didn't work because I am bad at cmake.

I'm comparing galleries to each other and getting a matrix of all "1"s. This is with the latest build committed about an hour ago.

Hi,

I am interested in working on computational demanding routines to accelerate them. I am looking into GPU(I think it would be great for this application) openMP and MPI.
Is there any particular issue at the moment with that?

We currently store template metadata in a QMap<QString,QVariant> table. This generic approach has proven quite successful for associating a wide variety of values with the enrolled template. Having said that, the current convention for storing landmarks and regions of interest is suboptimal. The current convention is as follows:

Template t;
QRectF face = ...; // calculate face location
t.file.insert("Face_X", face.x());
t.file.insert("Face_Y", face.y());
t.file.insert("Face_Width", face.width());
t.file.insert("Face_Height", face.height());
QPointF leftEye = ...; // calculate left eye location
t.file.insert("Left_Eye_X", leftEye.x());
t.file.insert("Left_Eye_Y", leftEye.y());

This convention dates back to when the metadata table was QMap<QString,float>. Since QVariant can store QPointF and QRectF directly, I suggest we switch to the following convention:

Template t;
QRectF face = ...; // calculate face location
t.file.insert("Face", face);
QPointF leftEye = ...; // calculate left eye location
t.file.insert("Left_Eye", leftEye);

In addition to reducing the number of keys in the table, this makes it much easier to implement File::landmarks() and File::ROIs() which return lists of the available landmarks and ROIs respectively.

We will still use File::anonymousLandmarks() and File::anonymousROIs() for scenarios where it is convenient not to associate a name with each landmark/ROI (and instead store lists of landmarks/ROIs in the keys "_Landmarks" / "_ROIs").

One other consideration is how circles will be stored. Since there is no QCircle class, I propose we store them as _QRectF) where rect.x = rect.y = radius.

Provided the proposed change is agreeable to everyone, I'll follow up with a patch that changes the convention throughout the code base. After that we can have the discussion about what metadata keys to standardize, thus formalizing how to write interoperable object detectors and landmarkers. I know at least @caotto and @mburge have expressed interest in seeing this happen.

I have a TemplateList of data and need to access the metadata for each template at a specific key. There currently is no functionality to do so (other then stepping through each file and pulling the metadata individually) Is this functionality that can be included?