how can i build the jni libs with x86 platform ?

anyone here to help i'm trying but its giving me an error.

Hello, I noticed there is release 1.5.4 available:
https://bintray.com/qualeams/Android-Face-Recognition-Deep-Learning-Library/Android-Face-Recognition-with-Deep-Learning-Library/1.5.4
from Jun 16, 2017, but it's not reflected in commits here, which are 3months old.

Could you please update this repo & let us know what fixes / improvements have been made in 1.5.4?
Thank you a lot!

I tried to recognise the face using the latest library build. But then , the mobile screen is black fully.

Im getting the following message in logcat.

06-10 16:28:58.578 8532-8542/ch.zhaw.facerecognition W/art: Suspending all threads took: 17.306ms
06-10 16:29:24.238 8532-8542/ch.zhaw.facerecognition W/art: Suspending all threads took: 8.277ms
06-10 16:29:35.658 8532-8548/ch.zhaw.facerecognition W/art: Suspending all threads took: 7.428ms
06-10 16:29:39.818 8532-8542/ch.zhaw.facerecognition W/art: Suspending all threads took: 9.988ms
06-10 16:29:40.508 8532-8548/ch.zhaw.facerecognition W/art: Suspending all threads took: 6.632ms
06-10 16:29:41.828 8532-8542/ch.zhaw.facerecognition W/art: Suspending all threads took: 15.030ms
06-10 16:29:49.358 8532-8542/ch.zhaw.facerecognition W/art: Suspending all threads took: 8.884ms

This occurs in most of my mobiles. May I know how to sort this out?

Hi, @sladomic ,thanks for your contribution very much.
I tried to get confidence score use "predict_probability=1",but when I set it , the library can't run ,because of the error"Model does not support probabiliy estimates". Can it be solved ? Thanks a lot!And do you think I can get confidence score with predict_probability(issue#13)? thank you!

Im trying to use this library on one of my project.But each time the app is crushed on this method.After lot of debugging i found a weird issue.This problem is occurred only when the app package name and the app application id is different.Any idea about that?

I am trying to transplant the facenet model trained in GPU to android as the part of the project does，unluckily，I got stuck as the original .pb file didn't work on the android for that the tensorflow things couldn't support some ops of the serialized model,I try to solve it by using the optimized_for_inference tools embedded in tensorflow ,and the bazel can play a same role by my understanding.but I failed。。
T^T
this project has the optimized_facenet.pb which is exactly what I need and it works!even the whole project doesn't work on my PC with the reason unknown.
I wander How you guys optimized the pb file so that it works on android things,please share it with me ,many many thanks!

Getting the following error in Lenovo mobile

05-21 16:42:59.568 7583-7725/com.eigenfaces E/cv::error(): OpenCV Error: Image step is wrong (The matrix is not continuous, thus its number of rows can not be changed) in cv::Mat cv::Mat::reshape(int, int) const, file /build/master_pack-android/opencv/modules/core/src/matrix.cpp, line 1047
05-21 16:42:59.568 7583-7725/com.eigenfaces E/org.opencv.core.Mat: Mat::n_1reshape__JII() caught cv::Exception: /build/master_pack-android/opencv/modules/core/src/matrix.cpp:1047: error: (-13) The matrix is not continuous, thus its number of rows can not be changed in function cv::Mat cv::Mat::reshape(int, int) const
05-21 16:42:59.568 7583-7725/com.eigenfaces E/AndroidRuntime: FATAL EXCEPTION: Thread-6776
Process: com.eigenfaces, PID: 7583
CvException [org.opencv.core.CvException: cv::Exception: /build/master_pack-android/opencv/modules/core/src/matrix.cpp:1047: error: (-13) The matrix is not continuous, thus its number of rows can not be changed in function cv::Mat cv::Mat::reshape(int, int) const
]
at org.opencv.core.Mat.n_reshape(Native Method)
at org.opencv.core.Mat.reshape(Mat.java:655)
at com.eigendetector.Recognition.Eigenfaces.addImage(Eigenfaces.java:179)
at com.eigenfaces.TrainingActivity$1.run(TrainingActivity.java:107)
at java.lang.Thread.run(Thread.java:818)

If ii want to detect the name of a person in separate string after recognized by a this application how can i do this one ?

how can i run open cv detection and recognition with camera on portrait mode ?

If you do it, you'll end up with a tingling problem.

studypage :
https://docs.opencv.org/3.4/d0/d6c/tutorial_dnn_android.html

logcat :
2021-05-16 01:56:36.396 27600-27600/com.example.yolov3opencvex E/OpenCV/StaticHelper: OpenCV error: Cannot load info library for OpenCV
2021-05-16 01:56:38.092 27600-27729/com.example.yolov3opencvex E/cv::error(): OpenCV(4.5.2) Error: Bad argument (The total number of matrix elements is not divisible by the new number of rows) in reshape, file /build/master_pack-android/opencv/modules/core/src/matrix.cpp, line 1186
2021-05-16 01:56:38.095 27600-27729/com.example.yolov3opencvex E/org.opencv.core.Mat: Mat::n_1reshape__JII() caught cv::Exception: OpenCV(4.5.2) /build/master_pack-android/opencv/modules/core/src/matrix.cpp:1186: error: (-5:Bad argument) The total number of matrix elements is not divisible by the new number of rows in function 'reshape'
2021-05-16 01:56:38.097 27600-27729/com.example.yolov3opencvex E/AndroidRuntime: FATAL EXCEPTION: Thread-4
Process: com.example.yolov3opencvex, PID: 27600
CvException [org.opencv.core.CvException: cv::Exception: OpenCV(4.5.2) /build/master_pack-android/opencv/modules/core/src/matrix.cpp:1186: error: (-5:Bad argument) The total number of matrix elements is not divisible by the new number of rows in function 'reshape'
]

error line :
detections = detections.reshape(1, (int) detections.total() / 7);

How do we deal with this?

I have used 'Tensorflow with SVM or KNN' algorithm for Face Recognition. It works fine

Can I please know if similar type algorithm exists for 'Tensorflow with SVM or KNN'
ie replace the tasks of SVM with DNN as I got to know that DNN performs better than SVM. So wanted to check it.
Please help me with it

Could not find ch.zhaw:facerecognitionlibrary:1.5.3

Is there any valid repository to import the library via gradle implementation?

I think in your PreferencesHelper.java file, the "facenet.pb" should be replaced with "optimized_facenet.pb" as there is no file named facenet.pb

public String getTensorFlowModelFile(){
        return sharedPreferences.getString("key_modelFileTensorFlow", "facenet.pb");
    }

should be

public String getTensorFlowModelFile(){
        return sharedPreferences.getString("key_modelFileTensorFlow", "optimized_facenet.pb");
    }

I find the following things

1. The recognition is working fine in the same lighting conditions as that used during training images
2. Recognition is not working fine if the training lighting conditions is even slightly different from the lighting conditions of training images. ( Assuming all training images are taken at 1 location)

After going through the code I find that RGB values of each pixel of the cropped face is given as input to tensorflow for feature extraction, which explains why the recognition is highly dependent on training images.

Can I please know how the training and recognition be made independent of Lighting conditions.
I am thinking of

1. Find out x,y coordinates of all the 68 facial Landmark Points (https://www.pyimagesearch.com/wp-content/uploads/2017/04/facial_landmarks_68markup.jpg).

Src : https://www.pyimagesearch.com/2017/04/03/facial-landmarks-dlib-opencv-python

https://hackernoon.com/building-a-facial-recognition-pipeline-with-deep-learning-in-tensorflow-66e7645015b8

2. Calculate the distance between adjacent points and store in a 1d array of size 68
   For eg
   Arr[0]= distance between x0,y0 and x1,y1
   Arr[1]= distance between x1,y1 and x2,y2
   Arr[2]= distance between x2,y2 and x3,y3
   .
   .
   .
   Arr[67]= distance between x67,y67 and x68,y68

3. Pass the Array Arr as input to TensorFlow for feature extraction and generate 128 bit feature vector.

The above method would

1. Be independent of Light as its independent of RGB values of each pixel.
2. The time taken by tensorflow would reduce as the input to tensorflow for 128 bit feature extraction would be reduced from 160 * 160 * 3 to 1*68

Could you please share your thoughts on it OR is there any other way to make face recognition independent of RGB values of each pixel

It is impossible to build a project with v1.5.3 dependancy as it fails as follows:

From Gradle sync (gradle:3.0.0-beta4'):
Unable to resolve dependency for ':app@debug/compileClasspath': Failed to transform file 'facerecognitionlibrary-1.5.3.aar' to match attributes {artifactType=android-exploded-aar} using transform ExtractAarTransform

Failed to transform file 'facerecognitionlibrary-1.5.3.aar' to match attributes {artifactType=android-exploded-aar} using transform ExtractAarTransform

java.io.EOFException: Unexpected end of ZLIB input stream
> Unexpected end of ZLIB input stream

I even downgraded Gradle to stable gradle:2.3.3 and it still has a problem with extracting v1.5.3 arr:
Error:Execution failed for task ':app:prepareChZhawFacerecognitionlibrary153Library'.

Unable to unzip 'C:....gradle\caches\modules-2\files-2.1\ch.zhaw\facerecognitionlibrary\1.5.3...\facerecognitionlibrary-1.5.3.aar' to ...

I cleared Gradle caches as well as project build multiple times, forced it to re-download dependencies & I still get this error. I tried to unzip the arr myself with the same result & a bit more debug info:
Headers Error
Unconfirmed start of archive
Warnings:
There are some data after the end of the payload data
CRC failed: assets\optimized_facenet.pb

Published v1.5.3 arr seems to be corrupted.

PS: I had no problems building with v1.5.4 utilizing older & newer Gradle as well as AS.
[UPDATE] Dependency v1.5.2 doesn't have this problem either, but lacks other fixes & build fails when it comes to NDK, so downgrading to v1.5.2 isn't an option either:
Error:Execution failed for task ':app:transformNativeLibsWithMergeJniLibsForDebug'.

More than one file was found with OS independent path 'lib/arm64-v8a/libopencv_java3.so'

if I want to store captured images on the server and while training images, I will fetch images from the server and then do recognition. For this what should I do and How?

OpenCV detectMultiScale mode has memory leaking. The apk's memory gradually increases more than 1G in this mode, then crash will happen.

The memory leaking seems to happen in OpenCV jni lib.

Could we get the source codes of OpenCV jni lib in order to find where the leaking is?

I have installed tensorflow in anaconda dir . I have trained and created a model now . I wish to optimize it.

I ran the following code

bazel build tensorflow/tools/graph_transforms:transform_graph

But it is firing the following error,

no such package 'tensorflow/tools/graph_transforms': BUILD file not found on package path.

CvException [org.opencv.core.CvException: cv::Exception: /build/master_pack-android/opencv/modules/core/src/matrix.cpp:1047: error: (-13) The matrix is not continuous, thus its number of rows can not be changed in function cv::Mat cv::Mat::reshape(int, int) const
]
at org.opencv.core.Mat.n_reshape(Native Method)
at org.opencv.core.Mat.reshape(Mat.java:655)
at ch.zhaw.facerecognitionlibrary.Recognition.Eigenfaces.addImage(Eigenfaces.java:178)
at com.prakshep.attendanceprakshep.trainingActivityPackage.TrainingActivity$1.run(TrainingActivity.java:106)
at java.lang.Thread.run(Thread.java:764)

how to resolve?

Document the code and add comments wherever necessary

Though I have turned on Eyes detection, it is not displaying any rectangle around my eyes and also it is not displaying right and left eye texts

Hi, can you please tell me how can I convert my .ckpt file to .pb file?
I want to use my own model.

Can I use the source code of facerecognitionlibrary. Then I can change the c/c++ codes according to my needs. Tks!

USER$ make
ndk-build
make[1]: Entering directory '/Users/nandita.damaraju/test/attempt13/Android-Face-Recognition-with-Deep-Learning-Library/jni-build'
[arm64-v8a] Compile++ : jnilibsvm <= common.cpp
jni/common.cpp: In function 'void cmdToArgv(std::string, std::vector<char*>&)':
jni/common.cpp:28:40: error: 'malloc' was not declared in this scope
char c = (char)malloc(c_size+1); // copy the null char in the end also
^
make[1]: *** [obj/local/arm64-v8a/objs/jnilibsvm/common.o] Error 1
make[1]: Leaving directory '/Users/nandita.damaraju/test/attempt13/Android-Face-Recognition-with-Deep-Learning-Library/jni-build'
make: *** [build] Error 2
USER$

Hi friends,
I m facing an issue if any new face will recognize and then given any old user name on that face.

Hello,
I have been using your library learning more about facial recognition, and I have noticed that the library requires me to use:
PreferenceManager.setDefaultValues(this, R.xml.preferences, false);
I understand that this sets the default values, but this preferences file is inside your library which points to your Test Framework Helper Classes, so I had to make my own preference XML file and helper class in order to set the default values. This one line was causing the issue like "error: (-13) The matrix is not continuous"

Can you please make this library independent of this preference file so that we do not have to make our own preferences XML file.

Thank you for this library though, it's amazing.

Hi,

I am finding it very difficult to understand how to use this library. I get that the necessary steps are training and recognition. Is there a simple example that shows how to do this? The library should be able to take a set of images for training (to add a person) and then an unknown image to try and recognize a known face in that picture. Please help me.

Thanks,
Sapnesh

Either build for CUDA and/or use the opencl repository

Im trying to convert tensorflow c++ files into so file

Following is the make file as in the project

build:
    ndk-build

install:
    cp -r libs/armeabi-v7a/*.so ../src/main/jniLibs/armeabi-v7a/
    cp -r libs/arm64-v8a/*.so ../src/main/jniLibs/arm64-v8a/

clean:
    ndk-build clean

But it is not building so file. What am I missing?

Hi, thanks for your work, I like it very much!
But I have some questions, does caffe or tensorflow can be used to train faces?
I see the code in caffe.java :

        caffe = new CaffeMobile();
        caffe.setNumThreads(4);
        caffe.loadModel(dataPath + modelFile, dataPath + weightsFile);

But I can not found the codes which create "modelFile" and "weightsFile" in the project.And they are not put in the project in advance.
Thank you!
@sladomic

Is it possible to train only the lastly added person instead of training from the scratch?

Hi,
There is a crash when I try to recognize face using Eigenfaces algorithm. The crash is in public String recognize(Mat img, String expectedLabel) method; to be more specific at the following line:

Core.subtract(img, Psi, img);

Here's error log:

CvException [org.opencv.core.CvException: cv::Exception: /Volumes/Linux/builds/master_pack-android/opencv/modules/core/src/arithm.cpp:639: error: (-209) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function void cv::arithm_op(cv::InputArray, cv::InputArray, cv::OutputArray, cv::InputArray, int, void (**)(const uchar*, size_t, const uchar*, size_t, uchar*, size_t, int, int, void*), bool, void*, int)
]
at org.opencv.core.Core.subtract_2(Native Method)
at org.opencv.core.Core.subtract(Core.java:2083)
at ch.zhaw.facerecognitionlibrary.Recognition.Eigenfaces.recognize(Eigenfaces.java:77)
at com.infolytx.facerecognizer.RecognitionActivity.onCameraFrame(RecognitionActivity.java:142)
at org.opencv.android.CameraBridgeViewBase.deliverAndDrawFrame(CameraBridgeViewBase.java:389)
at org.opencv.android.JavaCameraView$CameraWorker.run(JavaCameraView.java:357)
at java.lang.Thread.run(Thread.java:841)

Would you please help me to solve it? Thanks.

I tried to import the module, but get this error instead
ERROR: Could not find method leftShift() for arguments [android_release_aar_8aaffqenud6qsg8w8hrovmfvc$_run_closure7@64a53476] on task ':facerecognitionlibrary:generateRelease' of type org.gradle.api.DefaultTask.
Any advice for this? Thx

Image Reshaping with SVM Crashed

When I tried the Android-Face-Recognition-with-Deep-Learning-Library-Test App it crashed to Image Reshaping with SVM settings.

Why is minSdkVersion 21 (Android 5.0 Lollipop) please?

I checked what changed in API 21 and obvious suspects for me were:

• Support for OpenGL ES 3.1
• Android Extension Pack
• android.hardware.camera2 API
  but I haven't found any of them used in the lib.

When it comes to NDK Native APIs, I also checked what LOCAL_LDLIBS you utilize, but I haven't found any so recent that it would require API level 21.

I also checked dependencies and OpenCV should run on far lower APIs (however your linked OpenCV port has also minSdkVersion 21) and when it comes to Tensorflow, it's native libraries can run on API >= 14 device (only their demo that utilized camera2 API requires API >=21, as mentioned here https://github.com/tensorflow/tensorflow/tree/master/tensorflow/examples/android).

So why is the API >=21 required please?

In arduino, the dfplayer worked well and Raspberry connected to HC05 Bluetooth and it worked very well.

However, Raspberry and FDplayer are not allowed. Can you look at the chords?

`import serial
import time

MP3_PLAYBACK_SOURCE = 0x09
MP3_VOLUME = 0x06
MP3_PLAYBACK_SOURCE_TF = 1

global send
default_buf = [0x7E,0xFF, 0x06, 0x00, 0x00, 0x00, 0x00,0x00,0x00, 0xEF]
mp3_cmd_buf = [0x7E,0xFF, 0x06, 0x00, 0x00, 0x00, 0x00,0x00,0x00, 0xEF]

global ser
ser = serial.Serial(
port='/dev/ttyAMA1',
baudrate=9600,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize= serial.EIGHTBITS,
timeout=1
)

global loop
loop = False

def dfplayer_init():
mp3_send_cmd(MP3_PLAYBACK_SOURCE, 0, MP3_PLAYBACK_SOURCE_TF)
time.sleep(0.01)
mp3_send_cmd(MP3_VOLUME, 0, 30)
time.sleep(0.01)

def Volume(v):
mp3_send_cmd(MP3_VOLUME, 0, v)

def Play():
global loop
loop = True
mp3_send_cmd(0x12, 0, 1)

def NextPlay():
global loop
loop = True
mp3_send_cmd(0x01, 0, 0)

def Stop():
global loop
if loop:
loop = False
mp3_send_cmd(0x16, 0, 1)

def mp3_send_cmd(cmd, high_arg, low_arg):
mp3_cmd_buf[3] = cmd
mp3_cmd_buf[5] = high_arg
mp3_cmd_buf[6] = low_arg
checksum = MP3_checksum()
mp3_cmd_buf[7] = ((checksum >> 8) & 0x00FF)
print(mp3_cmd_buf[7])
mp3_cmd_buf[8] = (checksum & 0x00FF)
print(mp3_cmd_buf[8])
for i in range(10):
USART_Transmit(mp3_cmd_buf[i])
mp3_cmd_buf[i] = default_buf[i]

def MP3_checksum():
sum =0
for i in range(1,7):
sum += mp3_cmd_buf[i]
return -sum

def USART_Transmit(data):
global ser
print(chr(data))
ser.write(bytes(data))

dfplayer_init()
NextPlay()`

I am using 'Tensorflow with LibSVM' , it works fine with normal faces.
But when I test it with dark faces , the accuracy of recognition of dark faces doesn't seem to be good.

Can I please know if I need to modify any parameters in training/recognition so that accuracy of recognizing dark faces be improved without affecting the accuracy for normal faces

• Add OpenCV-android-sdk/sdk/native/libs/arm64-v8a/libopencv_java3.so
• Add "cp -r libs/arm64-v8a/*.so ../src/main/jniLibs/arm64-v8a/" to "jni-build/Makefile"
• Add "APP_ABI := armeabi-v7a arm64-v8a" in "jni-build/jni/Applikation.mk"
• Make
• Make install