[Plugin] Leap Motion - Event Driven

[getnamo]

@RalfRab;485466

Hello,

somehow I do not manage to see the leap motion image in my scene.
I have added a leap controller in my level BP and did enableImageSupport.
…
I have a cube in my scene with a billboard compontent. The sprite is scaled to screen size with factor 0.25.
…
I have added the key 1 event BP in order to see the leap motion image, but I actually is not showing up. Do you have any hints, what I might do wrong?

Thanks

Make sure your sprite isn’t set to hidden in game. While your bps look correct you can also try the EventRawImageReceived to get images.

@Ders92;485405

Again thank you for your answer it partly solved my issue as I wrote earlier. However the leap hands seems to get updated weirdly and not adapting to the change in “world to meters”. So I looked through the source code for your plugin and found a function you use called “convertAndScaleLeapToUE” in LeapInterfaceUtility.cpp that is used when translating the hands. So my thought is that perhaps the worldToMeters variable should be used in this function so the hands can adapt to the change in scale.

I tried to change the leap hands translation like this:
FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE * worldToMeters/100.0,
leapVector.x * LEAP_TO_UE_SCALE *worldToMeters/100.0 ,
leapVector.y * LEAP_TO_UE_SCALE * worldToMeters/100.0);

But this has no effect (tried with extreme values) and im suspecting its because I use blueprints and that the c++ files aren’t connected to the blueprints? So my question is if there is a way to achieve this in blueprints or do I have to use your plugin with c++, or is there another way to fix my issue?

Thanks in advance!

Still no better on this issue, I’m hoping someone else has a better understanding of what happens when you change scale. The thread for this problem is here: "World to meters" parameter problem - XR Development - Epic Developer Community Forums

@Opamp77;484950

It involves engine mod’s unfortunately.
Im using the same render thread delegate that I mentioned in the portal thread.
The delegate is an addition the the Headmounted display module.
A modifcation to the oculus plugin broadcast’s to the delegate the final eye pose for each camera when it is usually calculated.

the bone’s in this instance are staticmeshcomponents attached to hand actors.
At beginplay the LeapManager actor registers a callback function UpdateHandsOn_RT() with the delegate.

on GameThread Tick() the hand bones and update normally using the latest leap::frame at that point in time along with the results of GetHMDOrientationandPosition().

when the oculus plugin has calculated the final eye pose on the render thread it broadcast’s location,rotation and Eye.
This fire’s off the previously registed UpdateHandsOn_RT() in leaphandmanager.

This in turn calls ApplyLateUpdateTransform() on the sceneproxy’s of each of the hands staticmeshcomponents using the latest availible leap frame along with the left eye pose.

I believe ApplyLateUpdateTransform() was implimented in 4.10 to reduce the lag for motioncontrollers.

Until we have a way of accessing the HMD’s final pose on the render thread we are reduced to hacking the engine.
Unless there a mechanism allready in place that im not aware of.
But like I said Epic seem to have a branch named orion!?!

I’ll put the experiment up on if you want to have a look?
The reduced latency definately helps with hand prescence.

That’s an interesting approach, I’ll have to look into some of the bind points you suggested. I know Motion Controllers get a late update function call, but I don’t know if regular InputModules do too. There could be a way for the leap motion to pretend it’s a motion controller module to get this late update without engine fix. That could be a fairly sizable refactor.

Regarding the branch, how do you find these branches? I haven’t found a good way to browser UE branches on . Sounds fascinating either way… not that I would know anything about that :rolleyes: