[Plugin] Leap Motion - Event Driven

[selio]

Working with This and VR and things are going mostly ok, but when I rotate my head, the rigged hands don't follow properly. I'm using a subclass of the LeapRiggedCharacter, only adding a material billboard to the blueprint. the debug hands remain properly placed, but as I turn my head, the rigged hands and debug hands come further and further apart, and stop following altogether after about 90 degrees of turning. When I return to a normal forward direction, the hands come back to match the debug position. What am I missing here?

EDIT: Got some screenshots of the problem, presented below.

[theknoppix]

Using your plugin with 4.8 and everything seems work fine. Thanks for it.

But i have 2 questions.

1. Leap motion tracking is a little bit laggy compared to official one. Is there any way to improve it?

2. Moving working through player's forward vector. I want to make it rift's rotation like on first person template from official demo pack, but couldn't find how.

Thank you again for the plugin. Amazing work.

[Slayemin]

Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

I have tested both the Jenga level that getnano built as well as developed my own level.

Okay, that's a bit different. Are you still experiencing the black screen in VR?

For the VR hands, can you verify that the leap motion device goes into HMD mode? Can you try the leap motion apps in HMD mode without any issues?

Can you also verify that your Leap Motion device is not mounted upside down on your rift?

[dougb]

So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.

Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

I have tested both the Jenga level that getnano built as well as developed my own level.

[Slayemin]

Fantastic plug-in! Thanks so much getnamo!

One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

Any idea?

Thanks!

So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.

[dougb]

Fantastic plug-in! Thanks so much getnamo!

One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

Any idea?

Thanks!

[devel.bmad]

I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).

Ok, thanks! Unfortunately at the moment on Mac I cannot even build the engine from source due to this bug (mac is not top priority for Epic either, even the official Leap plugin is not yet available for mac in 4.8)

[Slayemin]

Okay, I've figured out my issues.

For anyone else who attempts to use a custom mesh and skeleton, make sure that the orientation of each bone exactly matches the orientation in the reference rig. Also, pay very close attention to bone weights on the fingers or you'll get messed up fingers.

The finger bones on my right hand were inverted along the X axis and the arm bones were correct, so changing the basis wasn't going to help fix the problem.

[getnamo]

Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

...

I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).

[devel.bmad]

That's because the precompile for 4.8 wasn't available yet. This has now been fixed.

Update to 0.9.4
-Updated Leap SDK to 2.2.6

Update to 0.9.5
-Updated binaries to UE4.8

get the newest master zip from github to upgrade.

Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

[Slayemin]

I'm glad you've asked the question, I was hoping someone would dig deep enough to ask about the implementation. The change basis function rotates all the AnimBody bones (that are parented, so if you rotate the body it will rotate all the bones) to a new basis. Pre-rotation applies input rotation before you're in the new space so it will affect all of your input rotations. In contrast post-rotation only rotates it by a constant amount and will not affect the basis of the input.

The reason for doing all of this is that in component space (e.g. character blueprints) X is forward Y is right, Z is up; but in animation Y is forward, X is left, Z is up. So in order for everything to appear correct we need to change the rotation space to the new Y = Forward orientation. That is all that function does.

Okay, I get the FABRIK and Two bone IK stuff. I'm still really stuck on the basis change stuff though.

I've got the left hand working perfectly, but I'm using different left prebase values and post base values:
Left Prebase: 90,0,0
Left Postbase: 0,0,0

I don't know why, but that works. Your values are different:
Left Prebase: -90,0,0
Left Postbase: 0,0,-90

The right hand is different though, and I'm stuck at the relationships between rotations:

I'm using the exact same values you're using within the LeapBodyConnector and I get wrong results:
Right Prebase: 90,0,180
Right Postbase: 0,0,90


Through some testing, I was able to find:
Right Prebase: -90,0,-180
Right Postbase: 0,0,0

This puts my elbow and palm in the correct position, but the right hand fingers are rotated 180 degrees (roll) in the wrong direction, and the thumb goes into the palm.

What's causing the difference in our pre and post basis values?

[getnamo]

Hi Getnamo,

While Inherting your plugin with recent rlease UE-4.8 raises error, [ATTACH=CONFIG]43826[/ATTACH]

How to rectify this, Exepecting ASP Getnamo. . .

That's because the precompile for 4.8 wasn't available yet. This has now been fixed.

Update to 0.9.4
-Updated Leap SDK to 2.2.6

Update to 0.9.5
-Updated binaries to UE4.8

get the newest master zip from github to upgrade.

[Rasa]

How to enable this plugin with UE-4.8

Hi Getnamo,

While Inherting your plugin with recent rlease UE-4.8 raises error,

How to rectify this, Exepecting ASP Getnamo. . .

[getnamo]

How to uninstall this plugin? I deleted all references, set the plugin to false in the config but still got a crash when restarting the project...

Inside your project root: Delete the LeapMotion plugin folder in Plugins and remove the leap specific binaries in your project (leap.dll, msvcr120.dll and msvco120.dll in Binaries/Win64 and Win32). Then it will be easy to clear the references from there.

[spyro]

How to uninstall this plugin? I deleted all references, set the plugin to false in the config but still got a crash when restarting the project...