Cannot Compile UE5 Pre2 + VS2019
The message I receive, while compiling (generated a new vs sln):

Gravità	Codice	Descrizione	Progetto	File	Riga	Stato eliminazione
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Avviso	C4996	'UTexture::Resource': Use GetResource() / SetResource() accessors instead. Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\DynamicTexture.cpp	95	
Avviso	C4996	'UTexture::Resource': Use GetResource() / SetResource() accessors instead. Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\DynamicTexture.cpp	98	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	C2440	'inizializzazione': impossibile convertire da 'const T *' a 'const float *'	MediaPipeDemo	C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\Plugins\MediaPipe\Source\MediaPipe\Private\MediaPipeShared.h	84	
Errore	MSB3073	uscita dal comando ""E:\Epic Games\UE_5.0\Engine\Build\BatchFiles\Rebuild.bat" MediaPipeDemoEditor Win64 Development -Project="C:\Users\dev\Desktop\ue4-mediapipe-plugin-master-ue5\MediaPipeDemo.uproject" -WaitMutex -FromMsBuild" con codice -1.	MediaPipeDemo	C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.MakeFile.Targets	51	

I have compiled the project and when I started to run the level the MediaPipe window appears black. I guess that is not connected to the webcam. Can you explain me where or in which config file can I address the webcam?

Best regards,

Mirko

Hey Wei Fong,

Is there a setting to turn off the rendering of the Landmarks in the video texture? I'd like to still use the Video without the landmark rendering. Or maybe is there somewhere I can use the video feed before it gets processed by mediapipe?

I cannot see where the webcam is accessed.
How would I implement this example to run on Android using the Phone's camera.

Good afternoon. I know that the system requirements are Windows, but I still tried to compile the project under Mac OS Mojave. The problem is that the compilation is successful, but when the project is launched there is no Pipeline component and all the components of your plugin. In the plugins editor, it is listed, but there are no functions in Blueprints. Is it possible to compile the plugin under MacOS?

In my project I need get the same webcamera so Can i Get the WebCamera Frame or data?

great job! When I test holistic landmarks demo, the landmarks are detected correctly, but they are not showed correctly in the right box.
Please look at the hand, the image is from front view but the points in right box are seem in side view.

Questions
where can I check the code that connects the landmark of the media pipe to the skeleton bones of the unreal engine? I want to learn that process! thanks

hello,this plgus is very good.

When I use the 3D detection of face key points, the mouth can be closed, but the eyes will not be closed. Have you encountered this kind of problem there?

Problem:Editor load stop at 75% Initializing...

Windows OS: Win11 Enterprise 22000.739

UE Version: UE5 5.0.2

VS Version:VS2022 MSVC v143 - VS 2022 C++ x64/x86 toolchain(Laster)

Log:
Display LogMediaPipe FMediaPipeModule::StartupModule
Display LogMediaPipe FindPattern Id=0 Status=0 Count=0 [(null)]
Display LogMediaPipe FindPattern Id=1 Status=0 Count=0 [(null)]
Warning LogMediaPipe FModuleTrace::OnDllLoaded NOT FOUND
Display LogMediaPipe PluginBaseDir: D:/WorkSpace/UEProject/mediapipe_/Plugins/MediaPipe
Display LogMediaPipe BinariesDir: D:/WorkSpace/UEProject/mediapipe_/Plugins/MediaPipe/Binaries/Win64
Display LogMediaPipe DataDir: D:/WorkSpace/UEProject/mediapipe_/Plugins/MediaPipe/ThirdParty/mediapipe/Data
Display LogMediaPipe GetDllHandle: D:/WorkSpace/UEProject/mediapipe_/Plugins/MediaPipe/Binaries/Win64/ump_shared.dll

log.txt
An error is reported at runtime logmediapipe: error: objectid out of range
The number of frames is greatly reduced (from 200 frames to about 60 frames)
The array length of 0 is the camera start-up time (or the face is not recognized)

Hi Wongfei,

Can I reach out personally to you, to ask how we could use MediaPipe for UE4.23.1?

Thank you for your time, it is valued.

大牛，你好，我这边有一些关于pbtxt方面的问题想资讯以你一下，不知道你放不方便留下联系方式，或者可以联系一下我的email：[email protected]或者[email protected]。
期待您的邮件

Hey wongfei,

Can I expect a plugin support for UE4.27? If so any idea when it might be released?
Thank you!

Is it possible to use this plugin to build Android/IOS app?

ERROR: Traceback (most recent call last):
File "C:/users/krishna/_bazel_krishna/mia4xlvz/external/build_bazel_rules_apple/apple/internal/ios_rules.bzl", line 529, column 58, in
ios_application = rule_factory.create_apple_bundling_rule(
File "C:/users/krishna/_bazel_krishna/mia4xlvz/external/build_bazel_rules_apple/apple/internal/rule_factory.bzl", line 950, column 55, in _create_apple_bundling_rule
rule_attrs.append(_common_binary_linking_attrs(rule_descriptor))
File "C:/users/krishna/_bazel_krishna/mia4xlvz/external/build_bazel_rules_apple/apple/internal/rule_factory.bzl", line 228, column 21, in _common_binary_linking_attrs
apple_common.objc_proto_aspect,
Error: 'apple_common' value has no field or method 'objc_proto_aspect'

i am using holistic with iris that can use all of stuffs of plugin. when i disable overlay, hand landmark doesn't work. pose_world_landmark is fine but hand is not working.

and is there anyway to convert output video to ue media texture?

Hi, is it possible to get video feeding before processing or to get the confidence of the hand finger? thank you

Hi, would can this plug-in detect tongue movement? thank you

I tried to change the camera resolution, but the mediapipe window appear to be same. Are there any settings in the code where I can change the input resolution, frame rate?

Originally posted by @PinkMediaGroup in #2 (comment)

If I try to open a project in UE 5.0.2, it will stop at 75% Initializing.

When I checked the log, I checked that there was no response after opening the ump_shared.dll file for the last time, but I'm not sure if this dll is really the problem.

How can I fix this?

Thank you for reading it.

Good afternoon. The last source. When packing a project under Windows, I can't start the resulting application. The application window freezes and nothing else happens. Even on an empty map. In the editor, everything starts and works like a charm.

Very good, can ump_shared.dll be open sourced?I want to try the plugin function on android.

It is a great plugin. ^^
I am thinking how to use it on my project.
My project is using MediaTexture with Mediaplayer as video source, So I wonder how can I link it to the plugin?

Thanks for your excellent job. I find the avatar's hand like thumb could not be controlled. Can I add the landmark by myself to control it in the mapping. I have tried it but it seems something wrong and avatar's thumb do not have any response. Do you know why?

Possible to provide UE5 Preview Binaries for mediapipe plugin? @wongfei

Hi, Could you release the privileges of the webcam? Currently, the webcam privilege is talking for your project and it is locked down. Is there any way to read the the webcam data or keypoint data? thank you

I was able to compile and build the plugin in UE4.26, it works well.

However when I package project to build an exe for Win64, with VS2019, a fatal error shows up:

fatal error LNK1169: one or more multiply defined symbols found
UATHelper: Packaging (Windows (64-bit)): Took 543.9567754s to run UnrealBuildTool.exe, ExitCode=6

before this, it has also some others LNK2005 errors.

Any thoughts on how to solve this?

Hi Wongfei,

I am currently looking into the code an dusage of the dlls and protobuf.

I am sure with some work this plugin could be edited to also include the MacOS platform, right?

And would it work to build to an HTML5 package?

Are you available for commission?

Thanks,

Aaron

Edit: I asked my colleague on Windows to open the project but he got the same error of not being able to build. He was on UE4.26.2.

Hi wongfei

My name is Anski, I am running a mobile game on ue4, and also need to apply mediapipe on it.
May I have your direct contact, so I can talk to you directly?

I really need your help, and consult.

many thanks!!!!

大佬你好！在运行本插件时，遇到如下问题：
单独的手部关键点检测（配置导入multi_hand_landmarks.pbtxt）是带手部检测置信度输出的，但是全身关键点检测（配置导入holistic_landmarks.pbtxt 或 holistic_with_iris.pbtxt）无手部检测置信度输出，项目要求需要手部检测置信度，该问题如何解决？

可否直接修改holistic_landmarks.pbtxt输出手部检测置信度，而不用改动代码的结构。

When I try Load ump_shared.dll,the engine stops at 75%.May I refer to your VisualStudio config?
Engine Version:UE5 Pre.
Thansks!

Thank you for the plugin, very nice job!.

As i ask, can you give some details or docs of how to implement another model like "objectron" from mediapipe into the plugin with c++? Best regards.

Hello, @wongfei , i am totally new to UE, ( i am familiar with mediapipe), recently i am getting started with learning UE, then i want to start with playing mediapipe in UE, then i find your this plugin, but i don't know how to install and use it in UE, could you please add more tutorial in README.md ? or tell me some related tutorial link ~

Hi! Wongfei,
When running with this plugin, I encountered the following problem:
The hand landmarks detection (configured by multi_hand_landmarks.pbtxt) output hand detection confidence, while the holistic landmarks detection (configured by holistic_landmarks.pbtxt or holistic_with_iris.pbtxt) does not output the hand detection confidence.
How to solve this issue? Can I just modify holistic_landmarks.pbtxt to add the corresponding output without changing the code structure.
Looking forward to your reply!

Is that possible to using this to drive metahuman charater? any way to convert 3d landmarks to pose in ue?

Visibility and presence should mean the same thing. I don't understand the difference between the two

Pose point has only position vector and no directional Euler angle, which leads to low efficiency in driving the model

I try to calculate by mathematical operation, but the effect is not very ideal.

Since last year, I have had the idea of porting mediapipe to fantasy. But my code is very bad. I spend more time using blueprints. Thank you very much for making this plug-in. I want to make it better

Is there in your plans to implement it ? Can you provide hints for our selves to program it ?

I am currently using your plugin alongside 2 others, OVRLipSync as well as JsonCsvToDataTableToStruct and they managed to package fine without your plugin in the project.

But when I include your plugin into the project it fails to do a successful package. Attached is the error log from when it failed to package.

Would you have any idea why this is happening? I'd appreciate all the help I can get!!

LogUObjectHash: Compacting FUObjectHashTables data took   1.13ms
LogMainFrame: Project requires temp target (SunPosition plugin is enabled)
LogLauncherProfile: Unable to use promoted target - ../../../../../../Users/MyComputer/Desktop/TestingProject/Binaries/Win64/UE4Game-Win64-Shipping.target does not exist.
UATHelper: Packaging (Windows (64-bit)): Running AutomationTool...
UATHelper: Packaging (Windows (64-bit)): Parsing command line: -ScriptsForProject=C:/Users/MyComputer/Desktop/TestingProject/MotionTrackTest.uproject BuildCookRun -nocompileeditor -installed -nop4 -project=C:/Users/MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -cook -stage -archive -archivedirectory=C:/Users/Dex-lab/Desktop/3-Combo-Plugin -pac
kage -ue4exe="C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\Win64\UE4Editor-Cmd.exe" -clean -compressed -ddc=InstalledDerivedDataBackendGraph -pak -prereqs -nodebuginfo -targetplatform=Win64 -build -clientconfig=Shipping -utf8output
UATHelper: Packaging (Windows (64-bit)): Setting up ProjectParams for C:\Users\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject
UATHelper: Packaging (Windows (64-bit)): MotionTrackTest.uproject requires a temporary target.cs to be generated (SunPosition plugin is enabled)
UATHelper: Packaging (Windows (64-bit)): ********** BUILD COMMAND STARTED **********
UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MotionTrackTest Win64 Shipping -Project=C:\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -Clean -NoHotReload C:\MyComputer/Desktop/TestingProject/MotionTrackTest.uproject -NoUBTMakefiles  -remoteini="C:
\Users\Dex-lab\Desktop\IslandHealth_POC" -skipdeploy -nobuilduht -log="C:\Users\MyComputer\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MotionTrackTest-Win64-Shipping.txt"
UATHelper: Packaging (Windows (64-bit)):   Cleaning OKI_AnimationTest binaries...
UATHelper: Packaging (Windows (64-bit)): Took 0.8157848s to run UnrealBuildTool.exe, ExitCode=0
UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe OKI_AnimationTest Win64 Shipping -Project=C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject  C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject -NoUBTMakefiles  -remoteini="C:\Users\Dex-lab\Desk
top\IslandHealth_POC" -skipdeploy -Manifest=C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Manifest.xml -NoHotReload -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-OKI_AnimationTest-Win64-Shipping_2.txt"
UATHelper: Packaging (Windows (64-bit)):   Using 'git status' to determine working set for adaptive non-unity build (C:\Users\Dex-lab\Desktop\IslandHealth_POC).
UATHelper: Packaging (Windows (64-bit)):   Parsing headers for OKI_AnimationTest
UATHelper: Packaging (Windows (64-bit)):     Running UnrealHeaderTool "C:\Users\Dex-lab\Desktop\IslandHealth_POC\OKI_AnimationTest.uproject" "C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Win64\OKI_AnimationTest\Shipping\OKI_AnimationTest.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors 
-abslog="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UHT-OKI_AnimationTest-Win64-Shipping_2.txt" -installed
UATHelper: Packaging (Windows (64-bit)):   LogInit: Display: Loading text-based GConfig....
UATHelper: Packaging (Windows (64-bit)):   Reflection code generated for OKI_AnimationTest in 4.8656366 seconds
UATHelper: Packaging (Windows (64-bit)):   Writing manifest to C:\Users\Dex-lab\Desktop\IslandHealth_POC\Intermediate\Build\Manifest.xml
UATHelper: Packaging (Windows (64-bit)):   Building OKI_AnimationTest...
UATHelper: Packaging (Windows (64-bit)):   Using Visual Studio 2019 14.29.30137 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133) and Windows 10.0.19041.0 SDK (C:\Program Files (x86)\Windows Kits\10).
UATHelper: Packaging (Windows (64-bit)):   [Upgrade]
UATHelper: Packaging (Windows (64-bit)):   [Upgrade] Using backward-compatible build settings. The latest version of UE4 sets the following values by default, which may require code changes:
UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     bLegacyPublicIncludePaths = false                 => Omits subfolders from public include paths to reduce compiler command line length. (Previously: true).
UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     ShadowVariableWarningLevel = WarningLevel.Error   => Treats shadowed variable warnings as errors. (Previously: WarningLevel.Warning).
UATHelper: Packaging (Windows (64-bit)):   [Upgrade]     PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs   => Set in build.cs files to enables IWYU-style PCH model. See https://docs.unrealengine.com/en-US/Programming/BuildTools/UnrealBuildTool/IWYU/index.html. (Previously: PCHUsageMode.UseSharedPCHs).
UATHelper: Packaging (Windows (64-bit)):   [Upgrade] Suppress this message by setting 'DefaultBuildSettings = BuildSettingsVersion.V2;' in OKI_AnimationTest.Target.cs, and explicitly overriding settings that differ from the new defaults.
UATHelper: Packaging (Windows (64-bit)):   [Upgrade]
UATHelper: Packaging (Windows (64-bit)):   Building 42 actions with 8 processes...
UATHelper: Packaging (Windows (64-bit)):     [1/42] optical_flow_field_data.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [2/42] anchor.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [3/42] classification.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [4/42] face_geometry.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [5/42] SharedPCH.Core.cpp
UATHelper: Packaging (Windows (64-bit)):     [6/42] MediaPipe.init.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [7/42] geometry_pipeline_metadata.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [8/42] MediaPipeModule.cpp
UATHelper: Packaging (Windows (64-bit)):     [9/42] MediaPipeObserverComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [10/42] MediaPipePipelineComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [11/42] MediaPipeLandmarkObserverComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [12/42] MediaPipePresenceObserverComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [13/42] matrix_data.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [14/42] DynamicTexture.cpp
UATHelper: Packaging (Windows (64-bit)):     [15/42] environment.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [16/42] image_file_properties.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [17/42] MediaPipeClassificationObserverComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [18/42] locus.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [19/42] mesh_3d.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [20/42] rasterization.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [21/42] MediaPipeClassificationObserverComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [22/42] rect.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [23/42] MediaPipeObserverComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [24/42] time_series_header.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [25/42] image_format.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [26/42] landmark.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [27/42] MediaPipePresenceObserverComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [28/42] Default.rc2
UATHelper: Packaging (Windows (64-bit)):     [29/42] OKI_AnimationTest.cpp
UATHelper: Packaging (Windows (64-bit)):     [30/42] MediaPipeFaceMeshObserverComponent.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [31/42] detection.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [32/42] location_data.pb.cc
UATHelper: Packaging (Windows (64-bit)):     [33/42] MediaPipePipelineComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [34/42] SharedPCH.Engine.cpp
UATHelper: Packaging (Windows (64-bit)):     [35/42] MediaPipeLandmarkObserverComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [36/42] Module.JsonCsvToDataTableToStruct.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [37/42] Module.OVRLipSync.gen.cpp
UATHelper: Packaging (Windows (64-bit)):     [38/42] Module.OVRLipSync.cpp
UATHelper: Packaging (Windows (64-bit)):     [39/42] MediaPipeFaceMeshObserverComponent.cpp
UATHelper: Packaging (Windows (64-bit)):     [40/42] Module.JsonCsvToDataTableToStruct.cpp
UATHelper: Packaging (Windows (64-bit)):     [41/42] OKI_AnimationTest-Win64-Shipping.exe
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::VerifyVersion(int,int,char const *)" (?VerifyVersion@internal@protobuf@google@@YAXHHPEBD@Z) already defined in webrtc.lib(common.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void * __cdecl google::protobuf::internal::ArenaImpl::AllocateAlignedAndAddCleanup(unsigned __int64,void (__cdecl*)(void *))" (?AllocateAlignedAndAddCleanup@ArenaImpl@internal@protobuf@google@@QEAAPEAX_KP6AXPEAX@Z@Z) already defined in webrtc.lib(arena.obj
)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "private: void __cdecl google::protobuf::Arena::OnArenaAllocation(class type_info const *,unsigned __int64)const " (?OnArenaAllocation@Arena@protobuf@google@@AEBAXPEBVtype_info@@_K@Z) already defined in webrtc.lib(arena.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::OnShutdownRun(void (__cdecl*)(void const *),void const *)" (?OnShutdownRun@internal@protobuf@google@@YAXP6AXPEBX@Z0@Z) already defined in webrtc.lib(common.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "protected: void __cdecl google::protobuf::internal::RepeatedPtrFieldBase::Reserve(int)" (?Reserve@RepeatedPtrFieldBase@internal@protobuf@google@@IEAAXH@Z) already defined in webrtc.lib(repeated_field.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "private: void __cdecl google::protobuf::internal::RepeatedPtrFieldBase::MergeFromInternal(class google::protobuf::internal::RepeatedPtrFieldBase const &,void (__cdecl google::protobuf::internal::RepeatedPtrFieldBase::*)(void * *,void * *,int,int))" (?MergeFromInte
rnal@RepeatedPtrFieldBase@internal@protobuf@google@@AEAAXAEBV1234@P81234@EAAXPEAPEAX1HH@Z@Z) already defined in webrtc.lib(repeated_field.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::InitSCCImpl(struct google::protobuf::internal::SCCInfoBase *)" (?InitSCCImpl@internal@protobuf@google@@YAXPEAUSCCInfoBase@123@@Z) already defined in webrtc.lib(generated_message_util.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "void __cdecl google::protobuf::internal::DestroyMessage(void const *)" (?DestroyMessage@internal@protobuf@google@@YAXPEBX@Z) already defined in webrtc.lib(generated_message_util.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > * __cdecl google::protobuf::internal::ArenaStringPtr::MutableNoArena(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const *)" (
?MutableNoArena@ArenaStringPtr@internal@protobuf@google@@QEAAPEAV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@PEBV56@@Z) already defined in webrtc.lib(arenastring.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ArenaStringPtr::DestroyNoArena(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const *)" (?DestroyNoArena@ArenaStringPtr@internal@protobuf@google@@QEAAXPEBV?$basic_string@DU?$
char_traits@D@std@@V?$allocator@D@2@@std@@@Z) already defined in webrtc.lib(arenastring.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: static unsigned __int64 __cdecl google::protobuf::internal::WireFormatLite::Int32Size(class google::protobuf::RepeatedField<int> const &)" (?Int32Size@WireFormatLite@internal@protobuf@google@@SA_KAEBV?$RepeatedField@H@34@@Z) already defined in webrtc.lib(w
ire_format_lite.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: __cdecl google::protobuf::internal::ExtensionSet::ExtensionSet(void)" (??0ExtensionSet@internal@protobuf@google@@QEAA@XZ) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: __cdecl google::protobuf::internal::ExtensionSet::~ExtensionSet(void)" (??1ExtensionSet@internal@protobuf@google@@QEAA@XZ) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::Clear(void)" (?Clear@ExtensionSet@internal@protobuf@google@@QEAAXXZ) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::MergeFrom(class google::protobuf::internal::ExtensionSet const &)" (?MergeErrorFrom@ExtensionSet@internal@protobuf@google@@QEAAXAEBV1234@@Z) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: void __cdecl google::protobuf::internal::ExtensionSet::Swap(class google::protobuf::internal::ExtensionSet *)" (?Swap@ExtensionSet@internal@protobuf@google@@QEAAXPEAV1234@@Z) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: bool __cdecl google::protobuf::internal::ExtensionSet::IsInitialized(void)const " (?IsInitialized@ExtensionSet@internal@protobuf@google@@QEBA_NXZ) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: unsigned __int64 __cdecl google::protobuf::internal::ExtensionSet::ByteSize(void)const " (?ByteSize@ExtensionSet@internal@protobuf@google@@QEBA_KXZ) already defined in webrtc.lib(extension_set.obj)
UATHelper: Packaging (Windows (64-bit)):     libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: static unsigned __int64 __cdecl google::protobuf::internal::WireFormatLite::UInt32Size(class google::protobuf::RepeatedField<unsigned int> const &)" (?UInt32Size@WireFormatLite@internal@protobuf@google@@SA_KAEBV?$RepeatedField@I@34@@Z) already defined in w
ebrtc.lib(wire_format_lite.obj)
UATHelper: Packaging (Windows (64-bit)):        Creating library C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.lib and object C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.exp
UATHelper: Packaging (Windows (64-bit)):     C:\Users\Dex-lab\Desktop\IslandHealth_POC\Binaries\Win64\OKI_AnimationTest-Win64-Shipping.exe : fatal error LNK1169: one or more multiply defined symbols found
UATHelper: Packaging (Windows (64-bit)): Took 57.1001483s to run UnrealBuildTool.exe, ExitCode=6
UATHelper: Packaging (Windows (64-bit)): UnrealBuildTool failed. See log for more details. (C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-OKI_AnimationTest-Win64-Shipping_2.txt)
UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=6 (6)
UATHelper: Packaging (Windows (64-bit)): BUILD FAILED
PackagingResults: Error: Unknown Error

Hello, I'm having a lot of fun with this plugin, however
It would be useful to be able to know which hand, left or right, each detection is.
The problem I'm having at the moment is that it starts out detecting both my hands, one of them is detection 1 and the other is detection 2, but if it looses detection of hand 1, then detection 2 becomes detection 1 since there is only one detection, which means anything that was using the landmarks of the previous detection 1 is teleported to the landmarks of the new detection 1. If I knew the handedness of each detection then I could tell anything using the landmarks to pay attention to handedness rather than detection order of the hands.

Can I detect multiple faces in the camera? If so, please tell me how?

Hello! I am nearing my project's end using this awesome plugin, but I'm facing a slight issue wrapping things up.

I don't want the Mediapipe window showing the raw camera feed to pop up on the final packaged project.
Turning the 'enable overlay' variable to false seems to do that, but it greatly reduces the performance of the hand landmark tracking.

My intended outcome is simply to prevent the extra window showing the webcam feed from popping up.
Am I doing this the wrong way?

Thank you.

Hi Wong Fei,

I've tested packaging with only the mediapipe plugin aswell as only the Oculus VR Lip Sync plugin and both are able to be packaged fine on their own. However trying to package them together produces an error that my colleague and I haven't been able to figure out.

If you have any idea why this is happening know of a fix it would be greatly appreciated.

(It seems that Github will wrap the text to fit so it's still difficult to read even when marked as code, sorry)
The Output Log for the Packaging Process:

LogUObjectHash: Compacting FUObjectHashTables data took 0.80ms LogMainFrame: Selected target: MediaPipeDemo LogLauncherProfile: Unable to use promoted target - D:/Unreal Projects/ue4-mediapipe-plugin/Binaries/Win64/MediaPipeDemo-Win64-Shipping.target does not exist. UATHelper: Packaging (Windows (64-bit)): Running AutomationTool... UATHelper: Packaging (Windows (64-bit)): Parsing command line: -ScriptsForProject="D:/Unreal Projects/ue4-mediapipe-plugin/MediaPipeDemo.uproject" BuildCookRun -nocompileeditor -installed -nop4 -project="D:/Unreal Projects/ue4-mediapipe-plugin/MediaPipeDemo.uproject" -cook -stage -archive -archivedirectory="D:/Unreal Projects/MediaPipeBuild" -package -ue4exe ="C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\Win64\UE4Editor-Cmd.exe" -clean -compressed -ddc=InstalledDerivedDataBackendGraph -pak -prereqs -distribution -nodebuginfo -targetplatform=Win64 -build -target=MediaPipeDemo -clientconfig=Shipping -utf8output UATHelper: Packaging (Windows (64-bit)): Setting up ProjectParams for D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject UATHelper: Packaging (Windows (64-bit)): ********** BUILD COMMAND STARTED ********** UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MediaPipeDemo Win64 Shipping -Project="D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -Clean -NoHotReload "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -NoUBTMakefiles -remoteini="D:\Unreal Proj ects\ue4-mediapipe-plugin" -skipdeploy -distribution -nobuilduht -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping.txt" UATHelper: Packaging (Windows (64-bit)): Cleaning MediaPipeDemo binaries... UATHelper: Packaging (Windows (64-bit)): Took 1.1431025s to run UnrealBuildTool.exe, ExitCode=0 UATHelper: Packaging (Windows (64-bit)): Running: C:\Program Files\Epic Games\UE_4.27\Engine\Binaries\DotNET\UnrealBuildTool.exe MediaPipeDemo Win64 Shipping -Project="D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" -NoUBTMakefiles -remoteini="D:\Unreal Projects\ue4-mediapipe- plugin" -skipdeploy -distribution -Manifest="D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Manifest.xml" -NoHotReload -log="C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping_2.txt" UATHelper: Packaging (Windows (64-bit)): Using 'git status' to determine working set for adaptive non-unity build (D:\Unreal Projects\ue4-mediapipe-plugin). UATHelper: Packaging (Windows (64-bit)): Parsing headers for MediaPipeDemo UATHelper: Packaging (Windows (64-bit)): Running UnrealHeaderTool "D:\Unreal Projects\ue4-mediapipe-plugin\MediaPipeDemo.uproject" "D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Win64\MediaPipeDemo\Shipping\MediaPipeDemo.uhtmanifest" -LogCmds="loginit warning, logexit warning, logdatabase error" -Unattended -WarningsAsErrors -abslog="C:\User s\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UHT-MediaPipeDemo-Win64-Shipping_2.txt" -installed UATHelper: Packaging (Windows (64-bit)): LogInit: Display: Loading text-based GConfig.... UATHelper: Packaging (Windows (64-bit)): Reflection code generated for MediaPipeDemo in 4.7631709 seconds UATHelper: Packaging (Windows (64-bit)): Writing manifest to D:\Unreal Projects\ue4-mediapipe-plugin\Intermediate\Build\Manifest.xml UATHelper: Packaging (Windows (64-bit)): Building MediaPipeDemo... UATHelper: Packaging (Windows (64-bit)): Using Visual Studio 2019 14.29.30137 toolchain (C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133) and Windows 10.0.19041.0 SDK (C:\Program Files (x86)\Windows Kits\10). UATHelper: Packaging (Windows (64-bit)): Building 42 actions with 8 processes... UATHelper: Packaging (Windows (64-bit)): [1/42] mesh_3d.pb.cc UATHelper: Packaging (Windows (64-bit)): [2/42] classification.pb.cc UATHelper: Packaging (Windows (64-bit)): [3/42] landmark.pb.cc UATHelper: Packaging (Windows (64-bit)): [4/42] locus.pb.cc UATHelper: Packaging (Windows (64-bit)): [5/42] environment.pb.cc UATHelper: Packaging (Windows (64-bit)): [6/42] time_series_header.pb.cc UATHelper: Packaging (Windows (64-bit)): [7/42] rect.pb.cc UATHelper: Packaging (Windows (64-bit)): [8/42] MediaPipePresenceObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [9/42] MediaPipeObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [10/42] detection.pb.cc UATHelper: Packaging (Windows (64-bit)): [11/42] image_file_properties.pb.cc UATHelper: Packaging (Windows (64-bit)): [12/42] rasterization.pb.cc UATHelper: Packaging (Windows (64-bit)): [13/42] MediaPipeClassificationObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [14/42] MediaPipeLandmarkObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [15/42] matrix_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [16/42] MediaPipePipelineComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [17/42] anchor.pb.cc UATHelper: Packaging (Windows (64-bit)): [18/42] image_format.pb.cc UATHelper: Packaging (Windows (64-bit)): [19/42] MediaPipeClassificationObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [20/42] MediaPipeFaceMeshObserverComponent.cpp LogUdpMessaging: Warning: FUdpMessageProcessor::UpdateReassemblers Discarding 1/2 of stale message segements from Sender=10.235.10.219:63482, Sequence=0 UATHelper: Packaging (Windows (64-bit)): [21/42] geometry_pipeline_metadata.pb.cc UATHelper: Packaging (Windows (64-bit)): [22/42] location_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [23/42] MediaPipePipelineComponent.cpp UATHelper: Packaging (Windows (64-bit)): [24/42] MediaPipeFaceMeshObserverComponent.gen.cpp UATHelper: Packaging (Windows (64-bit)): [25/42] optical_flow_field_data.pb.cc UATHelper: Packaging (Windows (64-bit)): [26/42] MediaPipePresenceObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [27/42] Default.rc2 UATHelper: Packaging (Windows (64-bit)): [28/42] MediaPipeLandmarkObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [29/42] face_geometry.pb.cc UATHelper: Packaging (Windows (64-bit)): [30/42] MediaPipeModule.cpp UATHelper: Packaging (Windows (64-bit)): [31/42] MediaPipeObserverComponent.cpp UATHelper: Packaging (Windows (64-bit)): [32/42] MediaPipe.init.gen.cpp UATHelper: Packaging (Windows (64-bit)): [33/42] DynamicTexture.cpp UATHelper: Packaging (Windows (64-bit)): [34/42] SharedPCH.Engine.ShadowErrors.cpp UATHelper: Packaging (Windows (64-bit)): [35/42] Module.OVRLipSync.cpp UATHelper: Packaging (Windows (64-bit)): D:/Unreal Projects/ue4-mediapipe-plugin/Plugins/OVRLipSync/Source/OVRLipSync/Private/CookFrameSequenceAsync.cpp(7): fatal error C1083: Cannot open include file: 'Async.h': No such file or directory UATHelper: Packaging (Windows (64-bit)): [36/42] Module.OVRLipSync.gen.cpp UATHelper: Packaging (Windows (64-bit)): D:\Unreal Projects\ue4-mediapipe-plugin\Plugins\OVRLipSync\Source\OVRLipSync/Public/OVRLipSyncPlaybackActorComponent.h(26): fatal error C1083: Cannot open include file: 'Classes/Components/AudioComponent.h': No such file or directory UATHelper: Packaging (Windows (64-bit)): [37/42] MediaPipeDemo.cpp UATHelper: Packaging (Windows (64-bit)): [38/42] MediaPipeDemo.init.gen.cpp UATHelper: Packaging (Windows (64-bit)): [39/42] MediaPipeCppExample.cpp UATHelper: Packaging (Windows (64-bit)): [40/42] MediaPipeCppExample.gen.cpp UATHelper: Packaging (Windows (64-bit)): Took 65.7694112s to run UnrealBuildTool.exe, ExitCode=6 UATHelper: Packaging (Windows (64-bit)): UnrealBuildTool failed. See log for more details. (C:\Users\Dex-lab\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.27\UBT-MediaPipeDemo-Win64-Shipping_2.txt) UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=6 (6) UATHelper: Packaging (Windows (64-bit)): BUILD FAILED PackagingResults: Error: Unknown Error

Helllo! Can i disable how can I turn off the drawing of landmarks on the camera. (it's like bones in Puppet avatar).

Hi. Thank you for your plugin. I can get the demo to work using BP, and everything works fine, but I am trying to use the plugin inside a custom C++ class, and I am having troubling identifying what to import and how to setup the equivalent BP code in C++ (I am trying to get 3D pose landmarks in C++). Is there some info on how to setup this pipeline, or can you provide some insight for C++? Thanks in advance. Best regards.

Multiple attempts failed to compile successfully

I know that Android isn't being implemented but
I want to run this plugin on Android.
Is it very difficult to run on Android?

is it possible to put an image from the camera in the UI?