OpenXRLeapProvider not loaded about unityplugin HOT 17 CLOSED

Hi @tomh4

The OpenXR content of the Ultraleap Tracking Plugin will only load if you also have the Unity OpenXR Plugin imported. Have you imported it via the Unity Package Manager? - its package name is com.unity.xr.openxr

from unityplugin.

Ah interesting @MattGrayUL, i only have the oculus open XR implementation loaded, isn't that sufficient?

from unityplugin.

I would expect so, as that should also need OpenXR

If you open the Package Manager window, do you have this?

from unityplugin.

No, as Oculus is using their own OpenXR implementation

"Note: The OpenXR backend mentioned in this topic is different than Unity OpenXR plugin."

• Reference

I started writing a custom OculusLeapProvider, but if this can be avoided by just "enabling" Oculus' OpenXR Runtime, that would be great

from unityplugin.

Ah yeah, our OpenXR implementation requires Unitys OpenXR Plugin, not Oculus'

We have an older Oculus-to-Leap provider available here if you wanted to try it out, or use it as a base. It worked with the Oculus Integration package on the Asset Store, but it was removed in January in favor of Unitys OpenXR implementation - it was from the Ultraleap Tracking Preview Package in version 6.5.0

from unityplugin.

@MattGrayUL thanks that will help,
would you be interested in the result? Should I share it? Or do you prefer keeping the OpenXR way ?

from unityplugin.

I think due to the way the majority of users use our Plugin, we will keep it using Unitys OpenXR Plugin, but it'd be great to know how your integration goes, to make sure anyone else asking about this gets the right response in the future :)

I will close this issue now though 👍

from unityplugin.

@MattGrayUL so far so good, this is were I get with the Oculus Provider:

As you can see, the thumb is offset as well as the palm itself.
Do you have any idea where I might look to improve this?

from unityplugin.

@MattGrayUL any ideas? I tried to play around with the offsets, but it is not really helping :(

from unityplugin.

I think this might relate to the way that Meta produce their hand data as we have seen mismatches between the data sets positions before

@rorygames may be better placed as he knows Physics Hands better than me though

from unityplugin.

From what I've been told, the Oculus hands don't currently conform to the OpenXR spec entirely which makes them report differently to what's expected. I'm not able to do a huge amount about this within the Physics Hands realm as I can't tell exactly what's currently being used (whether it's OpenXR Ultraleap hands vs OpenXR Oculus hands). When the hands conform better from all ends then things should be better (but I obviously don't know when that will happen on the Oculus side).

from unityplugin.

Thanks for the quick reply @rorygames
So from what I can see, we have a metacarpal thumb and pinky, and the wrist seems to have a new offset.
The wrist I can balance out, and by setting the thumb metacarpal to the thumbs proximal, i get a "somewhat" acceptable result but I would definitely want to improve on this.
I am currently using the vector hand to leap hand "Decode", and I would rather not change a lot in there as I guess this is used on other places as well, so I would like to modify my OculusProvider in such a way as to convert the hand correctly.

Do you have any documentation or similar which shows me what the expected input is?

from unityplugin.

I'm kind of amazed that that's working as it is currently 😅 (pretty sure VectorHand nukes a few variables that physics hands makes use of).

So by default we're expecting the hands to come in as basic OpenXR hands, which then get slightly modified by the openxr provider (to make them conform back to Leap version hands, idea being that if the data is conformant then we're all good).
The offsets are initial hand positions are set from a default hand pose we create off of our "TestHandFactory" script. You can see the flow of the offsets initial values by following the generation functions. Following it through you'll see a function called TestHandFactory.MakeTestHand which gives the default poses of the hand that you see in the editor outside of play mode.
You can also see that we offset the thumb angle within the Setup stage of the physics hand which is located in the PhysicsHandUtils. This also happens within the ResetPhysicsHandSizes function within the same script (which is called when the hand regains tracking).

I don't personally use a Quest (nor have spent much time with the hand tracking coming from it) but those areas should help you find what you might need to modify 🤞

from unityplugin.

So, I cannot expect VectorHand.Decode to produce a valid output?
In that case, I might take the OpenXRLeapProvider as starting point, and try to get the LeapFrames populated in a similar fashion.

Additionally , I unchecked "Match Joint Positions With Tracking Data" and now, the hand at least looks as expected, with a slight wrist offset which I am sure I can correct. However, due to the new thumb metacarpal the thumb behaves weirdly, so that is something I need to figure out:

from unityplugin.

VectorHand is designed for use within networking, so currently just reports the absolute minimal version of a hand that can be used for visual reconstruction but shouldn't be fully relied on for accuracy (it also reduces some variable precision which can result in a bit of jumping/rounding).

from unityplugin.

Okay thanks, so I will go the OpenXRLeapProvider way, I will report back if I make any progress :)

from unityplugin.

so im pretty close to what I want now @rorygames , but the thumb is still acting weird.

The rotation coming from oculus is splitted partially into the metacarpal of the thumb, and the proximal. I am not sure if the physics hand is correctly interpreting non-identity rotations of the 0-th bone of the thumb?

from unityplugin.