[Plugin] Leap Motion - Event Driven

[selio]

Just tried with the basicRiggedCharacter and RiggedCharacter and both of them behave the same as my version.

[malcriado]

So...leap motion in unreal continues to be a pain in the b*tt... When preview in vr, my character appears rotated 90 degrees from its head (hmd)...its been two years since leap motion came to the market and still no stable and solid develop for Unreal, and i dont want to do Unity pleaseeeee...any chance for a little bit of help?
Thanks in advance!

F.

[malcriado]

Screenshot that shows what i posted before

[Slayemin]

So...leap motion in unreal continues to be a pain in the b*tt... When preview in vr, my character appears rotated 90 degrees from its head (hmd)...its been two years since leap motion came to the market and still no stable and solid develop for Unreal, and i dont want to do Unity pleaseeeee...any chance for a little bit of help?
Thanks in advance!

F.

From what I can tell, that's not a "Leap Motion" problem -- it's a blueprint/content problem. Try rotating your character mesh by 90 degrees.

All leap motion does is it detects your hand and joint positions. What I found to be super helpful is to enable debug drawing mode so that you can view each of these joints as the leap motion device sees them. It's also very helpful to switch between the infrared view and the game view to make sure that your hand positions are tracking with the bone positions (which means you'll have to make fine tuned adjustments to the LM offset values in the character blueprint). The community plugin tries to map the leap motion joint orientations to your characters joint orientations (in anim blueprint), so if your orientations are off by 90 degrees, you're going to get wonky behavior.

I heard from an epic employee a few weeks ago that the sole guy who was working on the leap motion integration at Leap Motion is no longer working there, so their plugin support is kind of "on hold" at the moment. This community plugin is probably better. The unity team and leap motion teams have also spent a lot more time refining the drivers to work well with unity, so the performance issues and minor glitches we see in UE4 are mostly just a % market share and resource allocation problem.

Anyways, I was able to get leap motion working with UE4. It wasn't easy, but I got it and I learned a lot in the process.

[mindridellc]

Hi there, I'm having trouble getting the debug sphere hands to appear in my project. My project is slightly different in that it's based off of the flying game template instead of the first person template. I don't know if this would effect my use of your plugin in any meaningful way, but it's the only way I can think of that differs significantly from the very detailed implementation instructions that you've provided. Maybe it's because my Pawn is based off of the Plane BP? I don't know. If I plug in your provided character controller and model instead of using my blueprint stuff, I can see my hands controlling the arms and hands of the provided orange model perfectly. But if I try to implement the debug hands myself, I don't see anything when I stick my hands out in front of the Leap.

I've provided a screenshot of the section of my blueprint that deals with the Leap stuff. I'm using UE 4.7.6 and the version of your plugin that's recommended for it (0.9.2 I think?).

[Slayemin]

Hi there, I'm having trouble getting the debug sphere hands to appear in my project. My project is slightly different in that it's based off of the flying game template instead of the first person template. I don't know if this would effect my use of your plugin in any meaningful way, but it's the only way I can think of that differs significantly from the very detailed implementation instructions that you've provided. Maybe it's because my Pawn is based off of the Plane BP? I don't know. If I plug in your provided character controller and model instead of using my blueprint stuff, I can see my hands controlling the arms and hands of the provided orange model perfectly. But if I try to implement the debug hands myself, I don't see anything when I stick my hands out in front of the Leap.

I've provided a screenshot of the section of my blueprint that deals with the Leap stuff. I'm using UE 4.7.6 and the version of your plugin that's recommended for it (0.9.2 I think?).

The blueprint looks right. Put in some break points and see if the blueprint interface events are even getting triggered. There's a chance (and its been my experience) that the events are being handled elsewhere and you'd have to create an event dispatcher which sends those events off to any subscribers. First things first though, figure out if the events are getting triggered and where they're getting triggered.

[getnamo]

Working with This and VR and things are going mostly ok, but when I rotate my head, the rigged hands don't follow properly. I'm using a subclass of the LeapRiggedCharacter, only adding a material billboard to the blueprint. the debug hands remain properly placed, but as I turn my head, the rigged hands and debug hands come further and further apart, and stop following altogether after about 90 degrees of turning. When I return to a normal forward direction, the hands come back to match the debug position. What am I missing here?

EDIT: Got some screenshots of the problem, presented below.

...

This looks like you don't have the VRPlayerController set in your game mode.

Hi there, I'm having trouble getting the debug sphere hands to appear in my project. My project is slightly different in that it's based off of the flying game template instead of the first person template. I don't know if this would effect my use of your plugin in any meaningful way, but it's the only way I can think of that differs significantly from the very detailed implementation instructions that you've provided. Maybe it's because my Pawn is based off of the Plane BP? I don't know. If I plug in your provided character controller and model instead of using my blueprint stuff, I can see my hands controlling the arms and hands of the provided orange model perfectly. But if I try to implement the debug hands myself, I don't see anything when I stick my hands out in front of the Leap.

I've provided a screenshot of the section of my blueprint that deals with the Leap stuff. I'm using UE 4.7.6 and the version of your plugin that's recommended for it (0.9.2 I think?)....

In your custom pawn did you add both a Leap Component and a Leap Event Interface? Without a component your events wont fire.

[bullale]

Hey Getnamo,

Today in the twitch stream Nick Whiting said that UE 4.9 will have a device-agnostic motion controller interface. Have you heard or seen anything about that? I'd like me PSMove plugin to implement that interface if possible.

[getnamo]

Hey Getnamo,

Today in the twitch stream Nick Whiting said that UE 4.9 will have a device-agnostic motion controller interface. Have you heard or seen anything about that? I'd like me PSMove plugin to implement that interface if possible.

Had a chance to play with this Motion Controller binding last weekend when I was given an updated build for the HTCVive Jam in London.

It is very applicable to hydra/psmove/touch/lighthouse or where you are tracking one point for each hand. For example for the jam I spent the first 30min porting over my hydra plugin, recompiling and simply updating the Motion Controller positions that I had attached to my blueprint when hydras were detected and moving. This allowed me to test my early builds completely using the hydras and then walk over to the limited Vive kits and it would work the very same, saving heaps of debugging time.

I am curious how this can be done in C++ so that the blueprint adaptation I used won't be needed and importantly I don't believe it addresses more complicated inputs such as the leap motion or indirect positional input devices such as the Myo. There should me more thought on that and I should really get in touch with Nick Whiting on the subject. The jam was very informative and some of the things I learned will be used in future updates; e.g. pulling events from components similarly to how you do that for collision interaction, eliminating the requirement for setting an interface to your blueprint.

Overall device agnostic binding is something I deeply believe in, but I think it needs to go one step further and abstract a whole body skeleton, which all input devices would fill. From there you would extract the information you want, with convenience components for most common configs. This way it would include input which doesn't fit into the two hand controller paradigm as well as full mo-cap suits.

[bullale]

Are the Motion Controller bindings up on github anywhere?

[Cassio_90]

Hey getnamo, this is a great plugin!
For a project I want to use a MakeHuman model, however their UE4 skeleton is different and using the leap plugin as it is, everything is messed up.
Is there a way to use this skeleton somehow with your plugin or what would be my next step to get this working?

[getnamo]

Hey getnamo, this is a great plugin!
For a project I want to use a MakeHuman model, however their UE4 skeleton is different and using the leap plugin as it is, everything is messed up.
Is there a way to use this skeleton somehow with your plugin or what would be my next step to get this working?

I've had some success with animation re-targeting, though some bones may have to be set to animation instead of physics. I'm hoping to talk to epic on something simpler, though there's no eta yet. Worst case is that you copy the convenience animation blueprint, change the mesh used for the blueprint and change bone names for each of the anim graph actions.

Are the Motion Controller bindings up on github anywhere?

They should be available in the 4.9 preview. Haven't had a look at how to bind new plugins there yet.

[bullale]

They should be available in the 4.9 preview. Haven't had a look at how to bind new plugins there yet.

I found them. I was having trouble because I kept thinking I was looking for SteamController, when in fact I was looking for SteamVRController.

This commit contains most of the relevant changes but there were a few updates since then.

From what I can tell...
- you'll need "HeadMountedDisplay" in your list of module dependencies.
- the controller class will implement IMotionController
- I don't know how people using the plugin in C++ but wish the interface to be device agnostic will access the controller (I use GEngine->HMDDevice for the DK2 headset, but I don't know what to use for the motion controller.)

[suisso]

Hello,
Thanks a lot for your hard work. I have tried to install the plugin on 4.7, 4.8, and 4.9 (from Gthub, matching plugin to my version, or using "master for 4.8+) sometimes I have to create an empty C++ class in my project for the plugin load, which is fine.
I get the plugin listed in all versions, I follow all the steps of your video here:
https://www.youtube.com/watch?v=I39nohn-Fq0

But when I change the game mode to my own "jengaGameMode", it just plays as normal. (that would be point 3a.2 in your original post) I am not in first person, can't see the hands. Any idea why it would happen?
I am new to Unreal and Leap, so please forgive my lack of knowledge, but I am stuck on this and any help would be welcome.

Thanks!

[getnamo]

Hello,
Thanks a lot for your hard work. I have tried to install the plugin on 4.7, 4.8, and 4.9 (from Gthub, matching plugin to my version, or using "master for 4.8+) sometimes I have to create an empty C++ class in my project for the plugin load, which is fine.
I get the plugin listed in all versions, I follow all the steps of your video here:
https://www.youtube.com/watch?v=I39nohn-Fq0

But when I change the game mode to my own "jengaGameMode", it just plays as normal. (that would be point 3a.2 in your original post) I am not in first person, can't see the hands. Any idea why it would happen?
I am new to Unreal and Leap, so please forgive my lack of knowledge, but I am stuck on this and any help would be welcome.

Thanks!

"If you hit play and the game mode doesn't change then delete the ThirdPersonCharacter." -Sprout, youtube comment

Some templates have a character in the map which forces the player to use that character, simply delete them (this happens at 3:19 in the jenga video) and it will work.