Download

[Plugin] Leap Motion - Event Driven

Hello. We have been using your plugin, and it has been working great. However, we need to be able to scale the hands and move them further away from us (so that we are bigger), but after hours of trying we can not figure out how to do it. Is there a way to do this?

Hey. I would like to change the skeletal mesh from the yellow guy to a custom one. Is there any tutorials or docs on how to accomplish this? Been googling for hours without any result.

This is really cool. I’ve had some success with timewarp but it jitters like mad (I think overdriving is the only way to have it work, but it introduces the jitter). The most reliable way of getting tracking performance seems to be to re-factor the plugin for input device module with late update and attaching it to a camera component. Full timewarp though is still eluding me. I’ll share my version once it’s a bit cleaner.

I think that the right place should still be leap-ue4/LeapInterfaceUtility.cpp at master · getnamo/leap-ue4 · GitHub. E.g. multiplying the UE_TO_LEAP_SCALE by w2m inside the code, will try this approach next time I do a plugin pass.

This might be a load order problem, see if changing the load order inside leap-ue4/LeapMotion.uplugin at master · getnamo/leap-ue4 · GitHub would solve that (try predefault).

You can use the blueprint custom approach and do whatever you need with the data, or you can modify e.g. convertAndScaleLeapToUE in leap-ue4/LeapInterfaceUtility.cpp at master · getnamo/leap-ue4 · GitHub to modify the scale. Probably adding support for world2meter and custom scaling should be added to the plugin. If you make a modification that works, consider making a pull request in the plugin.

I’m still working on body-state which should make this easier in the future but for now the best approach is to use Animation Retargeting.

Hey getnamo

A completely crazy idea. would it be possible to get the leap motion hands working in the VR editor instead of the controllers with gestures as the buttons. See, told you it was a crazy idea :slight_smile:

Hi Getnamo, thanks for your plugin, we have had it working brilliantly. I was wondering if you are planning to update to 4.11 anytime soon? Unfortunately - trying and failing to be clever and ‘future proof’ we decided to build our project in 4.11 and now we want to integrate leap motion UE4 throws up a load of errors when we try to put the plugin in our project. Screenshots of the errors attached if they are of any help to you. thanks again!

0c22f8ecf5f566cd72b6acda2b73c1116fc66314.png
5f694a251863007119f258a533ebaf384339b8a0.png
Leap3.PNG

Hey,

I’ve encountered some problem using leapmotion + player teleportation/rotation.

I am working on a project that uses the oculus, to get a view of the backside we teleport the player to the new location and rotate him 180 degrees, so he looks straigth at the backside of the object.
The problem is, the controller (LeapEchoCollusionCharacter) doesn’t rotate with the player. That means the right hand always shows up, but the left hand doesn’t. It is positioned behind the player and you have to look at it first in order to be able to use it.

I need a workaround for this, 'cause we can’t have it this way.

Everything works now! This is what solves it in the function convertAndScaleLeapToUE:
float w2m = GWorld->GetWorldSettings()->AWorldSettings::WorldToMeters / 100.0f;
FVector vect = FVector(-leapVector.z * LEAP_TO_UE_SCALE * w2m, leapVector.x * LEAP_TO_UE_SCALE * w2m, leapVector.y * LEAP_TO_UE_SCALE * w2m);

this is something i tried at first but the thing i missed was that the size of the hand meshes has to be scaled with the same factor (w2m). So I just exported your hands to blender, scaled them and then reimported them and force replaced the old hands_lo and its skeleton with the new. I got a weird orientation error but it was solved by checking the box “TOas ref pose” under mesh settings when importing.

Hey getnamo, thanks for the reply! I tried adding the line

but I still have the same issue. The workaround is not really hindering me in any way and I would like to ask another question regarding the Leap and UMG.

I would like to have Leap interaction with a 3D menu that I have. I’ve managed to attach the menu to my custom leap character (subclassing LeapCollisionCharacter) because I want the menu to follow the camera movements of the user as shown below:

https://forums.unrealengine.com/attachment.php?attachmentid=84671&stc=1

However, I’m not entirely sure where to start in order to get the menu to interact with the hand. I found this link 3D cursor for OR 3d UMG - UE4 AnswerHub where you said you are using the same hack but I’m guessing instead of using the mouse position you use the hand position translated to (x,y) that you described here. Is it possible for you to put up the blueprint you were using or a tutorial on how to dit?

**EDIT: **I’ve tried first by adding OnComponentBeginOverlap(menu) as well as adding a box collider and using OnComponentBeginOverlap(box) to just print the location of the hit (breaking the hit result) but it’s not generating any overlap events.

EDIT2: So I figured out what was wrong, I had forgotten to enable “Generate Overlap Events” on the hand meshes and I can now get events when the hand overlaps with the menu widget, Now I’m looking at translating the 3d world location of the hand to the widgets local space as well as calling functions on the menu widget.

Thanks in advance!

I got a Problem with Changing the Parent Class of “Leap Basic Rigged Character” from Character(C++) to Spectator Pawn(C++). I just assigned everything exactly the same as before after Reparenting but Ingame my hands are not controlling the Skeletal mesh. Did I miss something here ?! I really don’t want my movement class to be Character, so have to change it to get 6-DOF kind of navigation.

Can i create custom gestures with this plugin?

Im am making a multiplayer game with leap motion however the hands of all players respond to the input of just one player, how can I fix this and make it individual to the client.

For example, I have my game running on two computers and when I put my hands in front of my face while on client 1, client 2’s player does the exact samething. It only does this for the leap input, using wasd to move is indeed isolated to the individual clients as it should be.

The plugin needs to be recompiled for 4.11 but presently that throws a lot of errors. I’m trying to fix them since I do need the plugin working on 4.11 soon. On the other hand, 4.11 final is officially out today so maybe this Leap plugin will now get some 4.11 love from the developers.

Cheers!

OK, it wasn’t that difficult. You just need to replace in the source all the SetFlags(RF_RootSet) calls with AddToRoot() and the HasAnyFlags(RF_RootSet) calls with IsRooted() and the plugin compiles. I did a quick test with a new 4.11 project and it seemed to work. Maybe Getnamo can confirm that this is correct and not just a hack.

Cheers!

It’s integrated directly in the engine now in 4.11 :slight_smile:

Just update your launcher version and enable the plugin. The interface is largely the same as before with a few new convenience characters and image support!

More thorough documentations on the new stuff incoming over the next few days.

Ah! great! Congratulations.

The new 4.11 integration works well. The only issue I’ve noticed is that I see warnings inside the floating hands character Blueprint:

Warning InputAction Event references unknown Action ‘ChangeImageHandTint’ for InputAction ChangeImageHandTint
Warning InputAction Event references unknown Action ‘ChangeHands’ for InputAction ChangeHands
Warning InputAction Event references unknown Action ‘ResetHMDOrientation’ for InputAction ResetHMDOrientation
Warning InputAction Event references unknown Action ‘ComfortTurnRight’ for InputAction ComfortTurnRight
Warning InputAction Event references unknown Action ‘ComfortTurnLeft’ for InputAction ComfortTurnLeft

Getnamo you’re my hero!

Just wanted to say that my Problem solved itself with the new Plugin that comes with 4.11 , great job!

That’s awesome, can’t wait for the documentation on the new stuff! Ever since I saw Orion I’ve been wanting to try it out.

Also i would like to ask if you could help me out with translating the fingers 3D world position to the widgets local 2D position? Currently I’m taking the rt_middle_a socket position and then using ProjectWorldLocationToWidgetPosition (I’ve also tried ConvertWorldLocationToScreenLocation which performed worse) to set the cursor image I have in my menu widget using SetPosition with the image as a Canvas Slot and I can get the image to move with the hand, but the image moves at a slower ratio than my hand so If I’m at the edges of my Menu then there’s a discrepancy between the finger position and the cursor image’s position. Any idea how I can solve it?

Thanks in advance!

Hi Getnamo,

I’m trying to Control Leap Echo hands Rigged Child Actor’s components from My_Character over blueprints . I tried to cast to it just to get the Left hand Mesh :

260e00f9dfb452eeb7f19863d3d65a59988afc7a.jpeg

But this always get failed , Am I doing something wrong ?!