Previewing Lip Tracker Freezes Parameters about vrcfacetracking HOT 4 CLOSED

I'll be removing the lip camera functionality in the next update for this very reason. The runtime really does not like me streaming live video from it. However, I may replace the image in the quickmenu for a live selfie view of your current avatar or simply a mirror to allow people to check if their face is tracking properly

from vrcfacetracking.

I did some testing to see if it actually see if this was on the SRanipal's end. I set up a basic Unity 2019.4.16f1 project, imported the ViveSR SDK, and added the Lip Framework Prefab.

I then setup 2 scripts: LipCameraViewer.cs and LipDataViewer.cs

I tried to set them up as close as you did, and LipData is v2. The two classes should be obvious, but I'll briefly explain what they do. LipCameraViewer will render the latest image from the tracker to a RawImage in a canvas. LipDataViewer will get the LipData and LipShapes/LipWeights and then output the TongueRoll property to a UI Text inside the same Canvas.

From my testing, I can see that running an update frame to get camera output while getting data does NOT effect anything on the SRanipal end at all. The data values seemed to still work and the camera output didn't lag or anything like that.

from vrcfacetracking.

Issue is, the mod has another step of invoking the functions on the il2cpp domain. This causes some instability and minor lag which I feel is probably to blame. My code essentially does the same as you've done in yours so I'm not quite sure what's doing it at this point

from vrcfacetracking.

Issue is, the mod has another step of invoking the functions on the il2cpp domain.

Changed the Scripting Backend to IL2CPP and set the Api Compatibility Level to .Net 4.x. The same results occur. TongueRoll seems to work properly and the images are still rendering properly, There seems to be no issues. If I need to, I can write a quick mod to link into VRChat and do the same thing for testing, but it wouldn't explain why it works fine in the Editor and not in-game.

A guess could be because you're modding it in with MelonLoader rather than building from unity (which I don't expect you to be able to do haha), but I don't see why that would make a difference.

I'm not quite sure what's doing it at this point

Here's my guess.
If you look at line 14 in LipTrackingMenuPage.cs ( url ), you can see it's setting the LipEnabled value to whatever b is. I think b is false which is causing all the parameters to stop working. This could explain why the parameters default to some value rather than freezing at what they last were. I don't know what event TrackingEnabled is, but it's odd how it wouldn't set the value back to true, maybe the true event never hits it because the class "ends?" If you can, I'd try registering the event somewhere else in a static class (like MainMod.cs) since the class LipTrackingMenuPage is non-static.

from vrcfacetracking.