这是一个 Flutter Packge 以实现摄像头精确追踪并识别十指的运动路径/轨迹和手势动作, 且输出22个手部关键点以支持更多手势自定义. 基于这个包可以编写业务逻辑将手势信息实时转化为指令信息: 一二三四五, rock, spiderman...还可以对不同手势编写不同特效. 可用于短视频直播特效, 智能硬件等领域, 为人机互动带来更自然丰富的体验
Home Page: https://zhzh.xyz/2020/03/01/Flutter/flutter-plugin-handtracking-landmark/
License: Apache License 2.0
Please, how can I get the hand_tracking_aar.aar file?
Hi,
First of all, thank you for spending time working up and publish this. This is very kind of you. I have tried this and it works great.
However, I would like to create another version for pose detection, I have re-compile the aar, copy the tflite and binary graph as you suggest. I also modify the NativeView to set the graph name and output, but when I run it, I got segfault like following:
E/DecorView(25629): mWindow.mActivityCurrentConfig is null
I/OpenGLRenderer(25629): onPositionUpdated - left: 0.000000, right: 1080.000000, top: 0.000000, botton: 1836.000000
I/OpenGLRenderer(25629): doUpdatePositionAsync is called and callVoidMethod
D/ExternalTextureConv(25629): Created output texture: 2 width: 720 height: 1280
I/native (25629): I20210716 11:18:11.265153 29341 graph.cc:474] Start running the graph, waiting for inputs.
I/native (25629): I20210716 11:18:11.265355 29341 gl_context_egl.cc:163] Successfully initialized EGL. Major : 1 Minor: 4
I/native (25629): I20210716 11:18:11.266983 29435 gl_context.cc:331] GL version: 3.2 (OpenGL ES 3.2 v1.r19p0-01rel0.###other-sha0123456789ABCDEF0###)
W/diapipe_exampl(25629): 0xebadde09 skipped times: 0
F/libc (25629): Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0 in tid 29427 (mediapipe/29427), pid 25629 (diapipe_example)
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
Build fingerprint: 'samsung/starltexx/starlte:10/QP1A.190711.020/G960FXXUFFUE1:user/release-keys'
Revision: '26'
ABI: 'arm64'
Timestamp: 2021-07-16 11:18:11+0700
pid: 25629, tid: 29427, name: mediapipe/29427 >>> co.sotai.flutter_mediapipe_example <<<
uid: 10672
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
Cause: null pointer dereference
x0 0000000000000000 x1 696c66742e6c6c75 x2 67616e616d5f6574 x3 0000000000000002
x4 0000000000000000 x5 0000006f070b2947 x6 20646c756f63203b x7 6461657220746f6e
x8 0101010101010101 x9 0000007fc9a03110 x10 000000000000000b x11 00000000fffffff5
x12 0000000000000041 x13 00000000fffffff5 x14 0000006f1f1dc5c2 x15 0000000000000054
x16 0000006f1f408158 x17 0000007030d22a60 x18 0000006ef829e000 x19 0000006f9b237840
x20 0000006f9b237860 x21 0000000000000000 x22 0000006f070c0a20 x23 0000000000000020
x24 0000006f9b238020 x25 0000006f9b237861 x26 0000000000000020 x27 0000006f070bfa00
x28 5441505f454c4946 x29 0000006f9b2377f0
sp 0000006f9b237740 lr 0000006f1eadd8bc pc 0000007030d22a70
backtrace:
#00 pc 000000000007fa70 /apex/com.android.runtime/lib64/bionic/libc.so (strlen+16) (BuildId: 3f350ca06c3b80560f65755286daf320)
#01 pc 00000000005bd8b8 /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::PathToResourceAsFile(std::__ndk1::basic_string<char, std::__
ndk1::char_traits<char>, std::__ndk1::allocator<char>> const&)+572)
#02 pc 00000000004d4c7c /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::LocalFileContentsCalculator::Open(mediapipe::CalculatorConte
xt*)+520)
#03 pc 00000000009ef7cc /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::CalculatorNode::OpenNode()+840)
#04 pc 00000000009dd6e0 /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::internal::SchedulerQueue::OpenCalculatorNode(mediapipe::Calc
ulatorNode*)+276)
#05 pc 00000000009dd47c /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::internal::SchedulerQueue::RunNextTask()+220)
#06 pc 0000000000a10c50 /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::ThreadPool::RunWorker()+392)
#07 pc 0000000000a10894 /data/app/co.sotai.flutter_mediapipe_example-en2_bY7qoV6mAkxsYJl_Cg==/lib/arm64/libmediapipe_jni.so (mediapipe::ThreadPool::WorkerThread::ThreadBody(void*)+1388)
#08 pc 00000000000e3b24 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36) (BuildId: 3f350ca06c3b80560f65755286daf320)
#09 pc 0000000000085330 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) (BuildId: 3f350ca06c3b80560f65755286daf320)
Do you have any idea what is it about and how to fix it? Actually, I can make the sample work with AAR and Java but never try with flutter before.
Thanks
when I pull the code from the repo and run it but there is an error.
Can someone help me, please. thanks in advance.
Execution failed for task ':app:processDebugResources'.
> Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
> Failed to transform file 'hand_tracking_aar.aar' to match attributes {artifactType=android-compiled-dependencies-resources}
> Execution failed for AarResourcesCompilerTransform: C:\Users\Son\.gradle\caches\transforms-2\files-2.1\f288be8ddaa4d301a851bc5959afacdc\jetified-hand_tracking_aar.
> C:\Users\Son\.gradle\caches\transforms-2\files-2.1\f288be8ddaa4d301a851bc5959afacdc\jetified-hand_tracking_aar\AndroidManifest.xml```
Hi. First of all, thanks for sharing good open-source.
I have cloned your repository, and tried to run the example on my local machine.
But when I tried to run it, I got errors like below.
Error code***
FAILURE: Build failed with an exception.
Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
Failed to transform file 'hand_tracking_aar.aar' to match attributes {artifactType=android-compiled-dependencies-resources}
> Execution failed for AarResourcesCompilerTransform: /home/daehyeon/.gradle/caches/transforms-2/files-2.1/f288be8ddaa4d301a851bc5959afacdc/jetified-hand_tracking_aar.
> /home/daehyeon/.gradle/caches/transforms-2/files-2.1/f288be8ddaa4d301a851bc5959afacdc/jetified-hand_tracking_aar/AndroidManifest.xml
Error code***
Since I have only experience with flutter but not aar, I do not know how to fix this error.
Do I need to follow your description from first to the end to run this package?
Thanks in advance.
Error downloading object: android/libs/hand_tracking_aar.aar (e9e16b6): Smudge error: Error downloading android/libs/hand_tracking_aar.aar (e9e16b67719b7f6a53ec6295764d4c5a05f25b09ddfdefb165da2ebcfd60e9d2): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
When trying to clone the project, this happens:
> git clone https://github.com/zhouzaihang/flutter_hand_tracking_plugin.git
Cloning into 'flutter_hand_tracking_plugin'...
remote: Enumerating objects: 162, done.
remote: Counting objects: 100% (162/162), done.
remote: Compressing objects: 100% (129/129), done.
remote: Total 405 (delta 65), reused 91 (delta 4), pack-reused 243
Receiving objects: 100% (405/405), 40.66 MiB | 3.72 MiB/s, done.
Resolving deltas: 100% (114/114), done.
Downloading android/libs/hand_tracking_aar.aar (106 MB)
Error downloading object: android/libs/hand_tracking_aar.aar (e9e16b6): Smudge error: Error downloading android/libs/hand_tracking_aar.aar (e9e16b67719b7f6a53ec6295764d4c5a05f25b09ddfdefb165da2ebcfd60e9d2): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
Is there a way to download the .aar file anyways?
Or can I create it myself? Did you simply use the steps as described in your blog post to create the .aar file?
https://zhzh.xyz/2020/03/01/Flutter/flutter-plugin-handtracking-landmark/#创建一个-mediapipe-aar
or could you share your BUILD file?
Update to V2
Seem to clone this repo cannot succeed with below issue:
This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
Thank you for sharing your code.
Is there any apk file?
你好，我想在Python环境下应用这 gesture detection的功能。可是我不知道如何下手，我对flutter一窍不通，能否指点指点？
谢谢
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
Are you considering making this a package for flutter on https://pub.dev/?
I want build an flutter app for pose detection using blaze pose
pschuegr@glitchy ~/dev/flutter_hand_tracking_plugin (master) $ git lfs fetch
fetch: Fetching reference refs/heads/master
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
error: failed to fetch some objects from 'https://github.com/zhouzaihang/flutter_hand_tracking_plugin.git/info/lfs'
Thanks!
Error downloading object: android/libs/hand_tracking_aar.aar (e9e16b6): Smudge error: Error downloading android/libs/hand_tracking_aar.aar (e9e16b67719b7f6a53ec6295764d4c5a05f25b09ddfdefb165da2ebcfd60e9d2): batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
您好，下载hand_tracking_aar.aar文件一直报错，可以把这个文件发到我的邮箱吗，谢谢。
[email protected]
Hi I have cloned your repository, and tried to run the example but the app crashed in the emulator with no errors
I'm new to flutter so i hope you can help me
Thanks in advance.
@zhouzaihang, Thanks for taking the time to upload the file to baidu. Unfortunately, I cant seem to download the file since I cant create an account. If its possible can you upload on Google Drive / Microsoft OneDrive / DropBox / any other direct file sharing website for a day? Ill upload it on AndroidFileHost for you, which would help anyone who cant access baidu in the future. Thanks :)
Hi, I am not being able to run the project on Android
What I do:
cd example/
flutter run -d {android_device_id} -v
And this are the last lines of the output
[ ] > Task :app:processDebugResources FAILED
[ +96 ms] FAILURE: Build failed with an exception.
[ +4 ms] * What went wrong:
[ ] Execution failed for task ':app:processDebugResources'.
[ ] > Could not resolve all files for configuration ':app:debugRuntimeClasspath'.
[ ] > Failed to transform file 'hand_tracking_aar.aar' to match attributes {artifactType=android-compiled-dependencies-resources}
[ ] > Execution failed for AarResourcesCompilerTransform:
/Users/alfredorinaudo/.gradle/caches/transforms-2/files-2.1/f288be8ddaa4d301a851bc5959afacdc/jetified-hand_tracking_aar.
[ +1 ms] > /Users/alfredorinaudo/.gradle/caches/transforms-2/files-2.1/f288be8ddaa4d301a851bc5959afacdc/jetified-hand_tracking_aar/AndroidManifest.xml
[ ] * Try:
[ ] Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
[ +20 ms] * Get more help at https://help.gradle.org
[ +1 ms] BUILD FAILED in 49s
[ ] 25 actionable tasks: 6 executed, 19 up-to-date
[ +608 ms] Running Gradle task 'assembleDebug'... (completed in 50.6s)
[ +9 ms] Exception: Gradle task assembleDebug failed with exit code 1
[ +2 ms] "flutter run" took 58,413ms.
[ +9 ms]
#0 throwToolExit (package:flutter_tools/src/base/common.dart:14:3)
#1 RunCommand.runCommand (package:flutter_tools/src/commands/run.dart:607:7)
<asynchronous suspension>
#2 FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:977:18)
#3 _rootRunUnary (dart:async/zone.dart:1198:47)
#4 _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#5 _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#6 Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#7 Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#8 Future._completeWithValue (dart:async/future_impl.dart:529:5)
#9 _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#10 _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)
#11 RunCommand.usageValues (package:flutter_tools/src/commands/run.dart)
#12 _rootRunUnary (dart:async/zone.dart:1198:47)
#13 _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#14 _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#15 Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#16 Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#17 Future._completeWithValue (dart:async/future_impl.dart:529:5)
#18 _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#19 _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)
#20 AndroidDevice.isLocalEmulator (package:flutter_tools/src/android/android_device.dart)
#21 _rootRunUnary (dart:async/zone.dart:1198:47)
#22 _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#23 _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#24 Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#25 Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#26 Future._completeWithValue (dart:async/future_impl.dart:529:5)
#27 Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:567:7)
#28 _rootRun (dart:async/zone.dart:1190:13)
#29 _CustomZone.run (dart:async/zone.dart:1093:19)
#30 _CustomZone.runGuarded (dart:async/zone.dart:997:7)
#31 _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1037:23)
#32 _microtaskLoop (dart:async/schedule_microtask.dart:41:21)
#33 _startMicrotaskLoop (dart:async/schedule_microtask.dart:50:5)
#34 _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
#35 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:169:5)
[ +221 ms] ensureAnalyticsSent: 219ms
[ +1 ms] Running shutdown hooks
[ ] Shutdown hook priority 4
[ +3 ms] Shutdown hooks complete
[ +1 ms] exiting with code 1
I checked and hand_tracking_aar.aar is on flutter_hand_tracking_plugin-master/android/libs/hand_tracking_aar.aar
how to make it so it detects both hands? right now its only working with one i tried using the numhands in the kotlin file but its not working out right?
I built the hand tracking aar file along with binarypb according to the instruc. given in your readme . (mediapipe latest ver and 0.7.5.) .
Surface texture is printing in the console showing frames are created.
But due to the black screen we cannot detect anything.
FLutter version used 2.2.1
after putting the load() function in a BUILD file......
commands for new aar
bazel build -c opt --strip=ALWAYS --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a
//mediapipe/examples/android/src/java/com/google/mediapipe/apps/aar_example:mediapipe_hand_tracking.aar
command for binarypb
bazel build -c opt --config=android_arm64 mediapipe/examples/android/src/java/com/google/mediapipe/apps/handtrackinggpu:handtrackinggpu
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.