TengineKit, developed by OPEN AI LAB.
TengineKit is an easy-to-integrate face detection and face landmarks SDK. At present, it can run on various mobile phones at very low latency.
- Apk can be directly downloaded and installed on the phone to see the effect.
or
- scan code to download apk
- Provide best performance in mobile client
- Provide the simplest API in mobile client
- Provide the smallest package in mobile client
- face detection
- face landmarks
- face attributes like age, gender, smile, glasses
CPU | Time consuming | Frame rate |
---|---|---|
Kirin 980 | 14ms | 71fps |
Qualcomm 855 | 15ms | 67fps |
Kirin 970 | 17ms | 59fps |
Qualcomm 835 | 18ms | 56fps |
Kirin 710F | 19ms | 53fps |
Qualcomm 439 | 26ms | 38fps |
MediaTek Helio P60 | 27ms | 37fps |
Qualcomm 450B | 28ms | 36fps |
The latest release is available on Maven Central.
implementation 'com.tengine.android:tenginekit:1.0.2'
...
import com.tenginekit.FaceManager;
public class CameraActivity extends AppCompatActivity implements Camera.PreviewCallback{
private static final String TAG = "CameraActicity";
// camera preview width
protected int previewWidth;
// camera preview height
protected int previewHeight;
// content display screen width
public static float ScreenWidth;
// content display screen height
public static float ScreenHeight;
// nv21 data from camera
protected byte[] mNV21Bytes;
...
public void Init() {
mNV21Bytes = new byte[previewHeight * previewWidth];
/**
* init
* */
FaceManager.GetInstance().init(this,previewWidth, previewHeight, ScreenWidth, ScreenHeight,FaceManager.ImageFormat.YUV_NV21);
}
/**
* Callback for android.hardware.Camera API
*/
@Override
public void onPreviewFrame(final byte[] bytes, final Camera camera) {
if (isProcessingFrame) {
return;
}
isProcessingFrame = true;
try {
if (mNV21Bytes == null) {
...
Camera.Size previewSize = camera.getParameters().getPreviewSize();
previewHeight = previewSize.height;
previewWidth = previewSize.width;
Init();
...
}
} catch (final Exception e) {
MyLogger.logError(TAG, "onPreviewFrame: " + e);
return;
}
processImage();
}
@Override
public void onDestroy() {
super.onDestroy();
/**
* release
* */
FaceManager.GetInstance().release();
}
protected void processImage() {
...
FaceInfo[] faceInfos = FaceManager.GetInstance().getFaceInfo(mNV21Bytes);
if (faceInfos != null && faceInfos.length > 0) {
Rect[] face_rect = new Rect[faceInfos.length];
List<List<LandmarkInfo>> face_landmarks = new ArrayList<>();
for (int i = 0; i < faceInfos.length; i++) {
face_rect[i] = faceInfos[i].faceRect;
face_landmarks.add(faceInfos[i].landmarks);
}
// do something with face_rect, face_landmarks
}
...
}
}
Android
- Min Sdk Version 19
The located Function under com.tenginekit.FaceManager
.
- context: context in activity,
- preview_w: Camera preview width
- preview_h: Camera preview height
- screen_w: Screen width
- screen_h: Screen height
- format: Data format(YUV, RGB)
FaceManager.getInstance().init(Context context, int preview_w, int preview_h, float screen_w, float screen_h, ImageFormat format);
enum ImageFormat
{
YUV_NV21,
RGB,
RGBA,
GRAY
}
- imageData: Input data
FaceInfo[] info = FaceManager.getInstance().getFaceInfo(byte[] imageData);
- ori: Rotation angle (vertical to 0°)
- is_screen_rotate: whether the display area follows rotation
- screen_w: screen width
- screen_h: screen height
FaceManager.getInstance().setRotation(int ori, boolean is_screen_rotate, float screen_w, float screen_h);
- back:Whether it is a rear camera
FaceManager.getInstance().switchCamera(boolean back);
- FuncName: Function name
- state: Whether the additional function type is enabled
FaceManager.getInstance().changeFuncState(ExtraFunc FuncName, boolean state);
enum ExtraFunc
{
Attribution, // face attrubution Function
}
FaceManager.getInstance().release();
- FaceInfo
Parameter name | Parameter type | Comment |
---|---|---|
faceRect | Rect | Face rectangle |
landmarks | List | Face key point information 212 points |
xAngle | float | Human face x direction corner |
yAngle | float | Human face y direction corner |
zAngle | float | Human face z direction corner |
leftEyeClose | float | Left eye closure confidence 0~1 |
rightEyeClose | float | Right eye closure confidence 0~1 |
mouseClose | float | Mouth closure confidence 0~1 |
mouseOpenBig | float | Open mouth Big confidence 0~1 |
- FaceInfo -- Attribution
Parameter name | Parameter type | Comment |
---|---|---|
age | int | Age |
isMan | boolean | Is it male |
smile | int | Smile degree0~100 |
glasses | boolean | Whether to wear glasses |
beatyOfManLook | int | Face value from a male perspective |
beautyOfWomanLook | int | Face value from a female perspective |
- LandmarkInfo
Parameter name | Parameter type | Comment |
---|---|---|
x | float | Point x position |
y | float | Point y position |
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.READ_PHONE_STATE"/>
<uses-permission android:name="android.permission.CAMERA"/>
Android code under the "sample/Android" folder.
In setRotation of TengineKit Api, there are two parameters ori and is_screen_rotate, which are the rotation angle and whether to follow the screen rotation. Whether the android:screenOrientation parameter in the Manifest follows the screen can be set. Not setting this parameter is to follow the screen rotation.
This part is to get data from Camera, as the SDK input.
We use the vertical screen as an angle of 0 degrees. Since the data collected by the Android camera always deviates by 90, it is necessary to set + (-90) when setting the ori parameter. The actual rotation angle of Android is to add a function through the sensor Calculate to get. For details, see the example in the Demo project.
When rendering, it is rendered at an angle of 0°, which is the normal output that people see under normal circumstances. The Android part has Canvas and Opengl rendering. Using Opengl rendering can make your apk better.
About the use of TengineKit and face-related technical exchanges, you can join the following QQ groups(Group Answer:TengineKit):
- TengineKit communication QQ group: 630836519
- Scan to join group