[Plugin] Leap Motion - Event Driven

[devel.bmad]

That's because the precompile for 4.8 wasn't available yet. This has now been fixed.

Update to 0.9.4
-Updated Leap SDK to 2.2.6

Update to 0.9.5
-Updated binaries to UE4.8

get the newest master zip from github to upgrade.

Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

[getnamo]

Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

...

I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).

[Slayemin]

Okay, I've figured out my issues.

For anyone else who attempts to use a custom mesh and skeleton, make sure that the orientation of each bone exactly matches the orientation in the reference rig. Also, pay very close attention to bone weights on the fingers or you'll get messed up fingers.

The finger bones on my right hand were inverted along the X axis and the arm bones were correct, so changing the basis wasn't going to help fix the problem.

[devel.bmad]

I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).

Ok, thanks! Unfortunately at the moment on Mac I cannot even build the engine from source due to this bug (mac is not top priority for Epic either, even the official Leap plugin is not yet available for mac in 4.8)

[dougb]

Fantastic plug-in! Thanks so much getnamo!

One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

Any idea?

Thanks!

[Slayemin]

Fantastic plug-in! Thanks so much getnamo!

One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

Any idea?

Thanks!

So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.

[dougb]

So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.

Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

I have tested both the Jenga level that getnano built as well as developed my own level.

[Slayemin]

Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

I have tested both the Jenga level that getnano built as well as developed my own level.

Okay, that's a bit different. Are you still experiencing the black screen in VR?

For the VR hands, can you verify that the leap motion device goes into HMD mode? Can you try the leap motion apps in HMD mode without any issues?

Can you also verify that your Leap Motion device is not mounted upside down on your rift?

[theknoppix]

Using your plugin with 4.8 and everything seems work fine. Thanks for it.

But i have 2 questions.

1. Leap motion tracking is a little bit laggy compared to official one. Is there any way to improve it?

2. Moving working through player's forward vector. I want to make it rift's rotation like on first person template from official demo pack, but couldn't find how.

Thank you again for the plugin. Amazing work.

[selio]

Working with This and VR and things are going mostly ok, but when I rotate my head, the rigged hands don't follow properly. I'm using a subclass of the LeapRiggedCharacter, only adding a material billboard to the blueprint. the debug hands remain properly placed, but as I turn my head, the rigged hands and debug hands come further and further apart, and stop following altogether after about 90 degrees of turning. When I return to a normal forward direction, the hands come back to match the debug position. What am I missing here?

EDIT: Got some screenshots of the problem, presented below.

[Rasa]

Hi Getnamo,

How to use Is connected or Is services connected under Leap controller in BluePrint. I'm trying to check the connectivity of Leap Motion with PC in BluePrint....
How its....????

[Neltor]

Yes, I'm doing this on two separate computers. Two of everything, connecting over LAN. Movements are replicated properly, meaning Player 1 can move and Player 2 sees the him move. I'll keep digging but it sounds like this *should* work. Have you tested this scenario yourself? Or anybody else tried this? Perhaps I'm missing some configuration.

Did you ever figure this out. I'm seeing the same thing. Thanks.

[mayaping]

I moved my hands(or fingers) by a wide margin.However,the debug sphere only moves in a small range.What's the problem?
I used the rigged character and the latest plugin.What's more,I do as the github guides.

[Slayemin]

Hi Getnamo,

How to use Is connected or Is services connected under Leap controller in BluePrint. I'm trying to check the connectivity of Leap Motion with PC in BluePrint....
How its....????

1. Make sure you character has "LeapAnimChildComponent", and get the child of that component and cast it to your "AnimBodyConnector" blueprint (or whatever you rename it to). Set this to a variable.
2. Call "GetLeapController" on the "AnimBodyConnector" node. This gives you direct access to the leap controller. From here, you can access the "Frame" object for the leap.

What I do is I run a polling service which records the last connection state for the leap each frame. I then compare this last connection state against the current connection state and I know that if they don't match, then the leap motion connection state has changed. Either it was reconnected or it lost connection (USB unplugged?). I fire off a dispatcher event which let's any subscribers know of these two events and they can handle it however they want.

Here is my polling blueprint node:

[Slayemin]

Working with This and VR and things are going mostly ok, but when I rotate my head, the rigged hands don't follow properly. I'm using a subclass of the LeapRiggedCharacter, only adding a material billboard to the blueprint. the debug hands remain properly placed, but as I turn my head, the rigged hands and debug hands come further and further apart, and stop following altogether after about 90 degrees of turning. When I return to a normal forward direction, the hands come back to match the debug position. What am I missing here?

EDIT: Got some screenshots of the problem, presented below.

It looks like you're not accounting for the HMD orientation and offset correctly within your characters AnimBlueprint. I could be wrong, but that's my intuition. The shoulder or spine transform is not rotating with the head, so there's probably a transformation break in the AnimGraph.

Does the out of the box version work perfectly? If so, does creating a character which inherits from the OOTB version break? If not, does adding a material billboard break it?