[Plugin] Leap Motion - Event Driven

[getnamo]

Hi getnamo and hi everyone.

First, thank you for this amazing plugin. I've tried it out and it works very well out of the box ! This will open new possibilities with VR and I can't wait to see future project using it.

I'm currently working on a networked simulation and I'm trying to figure out how to replicate hands through the network.
I think the plugin is currently not setup to do so (correct me if I'm wrong), so I started to modify a bit the plugin blueprints.

I've managed to replicates a LeapRiggedEchoHandsActor and its sub components/actor but I'm stuck at animating it. For now, the leap motion client implements the HandMoved event and call a server RPC. The issue is that on the server side the LeapHand reference is not valid so I guess this one is not replicated. To do so I have to start modifying the C++ plugin code and this may be a bit harder than expected.

So before I run into this I would like to know if someone already managed to replicated the hands through the network.

Best regards.

Great investigation and I'm looking forward to what solution you come up with. Make sure to make a pull request when you get it working!


On a side note has anyone else had this issue: https://github.com/getnamo/leap-ue4/issues/12 in packaged games? If so what version of the plugin/editor did you see this?

[NymKappa]

Hi getnamo and hi everyone.

First, thank you for this amazing plugin. I've tried it out and it works very well out of the box ! This will open new possibilities with VR and I can't wait to see future project using it.

I'm currently working on a networked simulation and I'm trying to figure out how to replicate hands through the network.
I think the plugin is currently not setup to do so (correct me if I'm wrong), so I started to modify a bit the plugin blueprints.

I've managed to replicates a LeapRiggedEchoHandsActor and its sub components/actor but I'm stuck at animating it. For now, the leap motion client implements the HandMoved event and call a server RPC. The issue is that on the server side the LeapHand reference is not valid so I guess this one is not replicated. To do so I have to start modifying the C++ plugin code and this may be a bit harder than expected.

So before I run into this I would like to know if someone already managed to replicated the hands through the network.

Best regards.

[Andre Jones]

How to detect it my Palm is UP or DOWN and if it is Left or Right Hand?

[getnamo]

Is it impossible to work with the Unreal Engine plugin without a VR system? I have 4.12.4 and the latest version of Desktop V2 (and recently tried Orion too), but when I override the blueprint to be the one with the FloatingHands and hit play, nothing happens. I can still move with my mouse but no input is being read from my Leap Motion.

Desktop mode should still work. Just note that if you do not use VR mode it expects the leap motion on your desk facing up. Please see https://github.com/getnamo/leap-ue4#...n-default-mode for simple note on how to adjust your desktop offset.

You should be able to use the same technique, but you may have to get the child actor which holds the raw hands themselves for collision. E.g. in rigged floating hands you have two skeletal meshes, which may need collision checks for the interface to work. Let me know how this works out so I can add the complete solution to documentation.

Hi Getnamo, I have the same issue. Any chance you could give more details on how to get the child actor? Thanks!

I'll try to find some time to show an example of this in blueprints, stay tuned.

When I downloaded the plugin, there is not LeapCollisionCharacter or VRPlayerController in the menu for selecting gamemode. Im using UE 12.3. Have you removed this from the latest release? If so, why isn't the documentation updated?

Documentation has been updated and references to LeapCollisionCharacter (other characters have their collision on by default) and VRPlayerController (no longer needed) have been removed. Despite this being the official leap motion plugin, it's still a community plugin in that it's a side project I do in my own free time. This means resources may not be always available and it relies on people such as yourself to point out and make pull requests where possible

[BoggyLP]

When I downloaded the plugin, there is not LeapCollisionCharacter or VRPlayerController in the menu for selecting gamemode. Im using UE 12.3. Have you removed this from the latest release? If so, why isn't the documentation updated?

[Bucket Monkey]

[QUOTE=getnamo;555998]You should be able to use the same technique, but you may have to get the child actor which holds the raw hands themselves for collision. E.g. in rigged floating hands you have two skeletal meshes, which may need collision checks for the interface to work. Let me know how this works out so I can add the complete solution to documentation.

Hi Getnamo, I have the same issue. Any chance you could give more details on how to get the child actor? Thanks!

[HintOfBlu]

Is it impossible to work with the Unreal Engine plugin without a VR system? I have 4.12.4 and the latest version of Desktop V2 (and recently tried Orion too), but when I override the blueprint to be the one with the FloatingHands and hit play, nothing happens. I can still move with my mouse but no input is being read from my Leap Motion.

[ameygumaste]

You should be able to use the same technique, but you may have to get the child actor which holds the raw hands themselves for collision. E.g. in rigged floating hands you have two skeletal meshes, which may need collision checks for the interface to work. Let me know how this works out so I can add the complete solution to documentation.

Surely will try to make it work! Thanks

[getnamo]

Hey Getnamo,I finally got it working on my setup at home! Thanks for your support!!

I got pick up and drop working for the leap rigged character. Very happy about it!!! Can I use the same technique for the Leap Floating hands Pawn? I tried to get the on actor hit event work for the floating hands pawn but it seems to be not working with it. I have added the PickUpCharacterInterface to the floating hands pawn.

Also I want to know whether we can use both the HTC Vive controllers as well as the Leap Motion Controllers together using this plugin?

Again really appreciate your help!! Thanks!

You should be able to use the same technique, but you may have to get the child actor which holds the raw hands themselves for collision. E.g. in rigged floating hands you have two skeletal meshes, which may need collision checks for the interface to work. Let me know how this works out so I can add the complete solution to documentation.

Hi,

I am working on a Mac using Unreal 4.12.3.
But there is too many compiler error of blueprint, I spend some time try to fix it, but it seems impossible.

I want to know when would the Orion Beta(VR) for Mac released, Because my project is very urgent.
If you could send me the unstable version - that would amazing.

Thanks for your time.

dymscut@gmail.com

There are currently no Mac compatible HMDs so supporting VR for Mac at the moment wouldn't serve a purpose; but I did also check the latest Leap Motion SDK resources and they still have Orion as windows only =/. You're best bet would be to use the 4.10 plugin which has the v2 tracking.

[leo dym]

Hi,

I am working on a Mac using Unreal 4.12.3.
But there is too many compiler error of blueprint, I spend some time try to fix it, but it seems impossible.

I want to know when would the Orion Beta(VR) for Mac released, Because my project is very urgent.
If you could send me the unstable version - that would amazing.

Thanks for your time.

dymscut@gmail.com

[ameygumaste]

Note that the pickup/drop method described in the jenga tutorial is just one approach of doing this. Follow the tutorial closely it is likely that the object you're trying to grab either doesn't have physics enabled or doesn't have the toggle enabled that physics simulation generates hit events. All of these steps are at least touched upon in the videos. If you still have problems despite checking those things, give more details on your setup and what you're trying to do in detail.

Hey Getnamo,I finally got it working on my setup at home! Thanks for your support!!

I got pick up and drop working for the leap rigged character. Very happy about it!!! Can I use the same technique for the Leap Floating hands Pawn? I tried to get the on actor hit event work for the floating hands pawn but it seems to be not working with it. I have added the PickUpCharacterInterface to the floating hands pawn.

Also I want to know whether we can use both the HTC Vive controllers as well as the Leap Motion Controllers together using this plugin?

Again really appreciate your help!! Thanks!

[ameygumaste]

Hey Getnamo,

Thanks for replying, Will work on it over the weekend with a clear head...Will post updates next week. Thanks!

[getnamo]

Hey Getnamo,

I have been trying to follow your Jenga Game tutorial for 2 days. I have tried all possible combinations, tried using UE 4.11 and 4.12 with both the Simple rigged character pawn and floating hands pawn. But I cannot seem to get past the "Trying to grab" print string on part 2 of the tutorial. I want to simply grab and drop an item. But does not work with the method of using interface that you have shared. I thank you for sharing your work, but is there any other simpler method to get this to work? I have used clean project, I have tried different templates and overall just keep hitting a roadblock. Please let me know if there are any updates that I should use, any support would be really appreciated.

Thanks,

Amey Gumaste

Note that the pickup/drop method described in the jenga tutorial is just one approach of doing this. Follow the tutorial closely it is likely that the object you're trying to grab either doesn't have physics enabled or doesn't have the toggle enabled that physics simulation generates hit events. All of these steps are at least touched upon in the videos. If you still have problems despite checking those things, give more details on your setup and what you're trying to do in detail.

[ameygumaste]

Hey Getnamo,

I have been trying to follow your Jenga Game tutorial for 2 days. I have tried all possible combinations, tried using UE 4.11 and 4.12 with both the Simple rigged character pawn and floating hands pawn. But I cannot seem to get past the "Trying to grab" print string on part 2 of the tutorial. I want to simply grab and drop an item. But does not work with the method of using interface that you have shared. I thank you for sharing your work, but is there any other simpler method to get this to work? I have used clean project, I have tried different templates and overall just keep hitting a roadblock. Please let me know if there are any updates that I should use, any support would be really appreciated.

Thanks,

Amey Gumaste

[Mala_L]

Last supported Mac build is 4.10 atm grab it here: https://github.com/getnamo/leap-ue4. The Orion runtime that was included in 4.11 was windows only, but that may change as we move forward, I'm due for a 4.12 update (outside engine for now, not fully stable). I do want to have Mac supported in engine, but in meantime try out the older builds.

hey that worked! I still had a camera / character in the scene so it wasn't getting picked up. Silly mistake. Thanks so much for your help, you've been brill!