[Plugin] Leap Motion - Event Driven

[getnamo]

Hi,

ty for your plugin. I am trying to get the camera image of the LeapMotion Controller to do some processing with OpenCV.
I have added LeapMotion to my modules and I can compile.



But I found this here...

Is there any way to get images or frames to use OpenCV on it?

Ty, sry for bad english.

Answered here: https://github.com/leapmotion/LeapUnreal/issues/41. Use SetLeapPolicy to enable raw image streaming, then subscribe to the OnImageEvent on your leap component.

[angrywilson]

The current plugin found at https://github.com/leapmotion/leapunreal should work both as project plugin and engine plugin as it detects it's environment. The general way to rebuild the plugin is to include it in e.g. a ue4 C++ project under plugins and regenerating your .sln then recompiling, the plugin will recompile with it.

The same process is used for engine plugins except that you replace the plugin found under e.g.

UE_4.21/Engine/Plugins/Runtime/LeapMotion

and you copy your third party includes to

UE_4.21/Engine/Source/ThirdParty/Leap

and dlls to

UE_4.21/Engine/Binaries/ThirdParty/LeapMotion

I'm hoping we will have an updated leap motion plugin for 4.22 which should eliminate the engine plugin workaround, but can give no guarantees at this time.

I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.

[getnamo]

Hi Guys,

Trying to packaga the demo scene VRPickupdrop but I keep getting the error below in the screenshot?

ATHelper: Packaging (Lumin): mabu: error: D:/Unreal/ML_PL/Intermediate/Lumin/Mabu\manifest.xml: missing <application ml:min_api_level=...> attribute

Anyone knows how to fix it please?

Tried to edit xml file but with ml:min_api_level="1" but it won't save down

Many Thanks

Lumin platform is Magic Leap, not Leap Motion. Wrong plugin!

[getnamo]

I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.

It's already in the engine (it has finally been updated to the modern architecture!), remove the project plugin and enable the engine version. There will be a project version release available within a week of when 4.22 releases.

[getnamo]

Hey Getnamo, I was wondering if it would be possible to retarget the leap motion hands to a full body skeleton mesh's hands? I would like to use leap motion for 3D animation,

I found this article: https://www.searis.net/assets/download/WilsonEtAl.pdf

It looks like it is possible, but I don't know if this is too advanced for me yet. I guess i will order one now since the alternative is spending over $1000 for gloves.

Thank you.

There is an example of this kind of rigging for the unreal mannequin, found here: https://github.com/leapmotion/LeapUnrealmodules#rigging

[AydinMAD]

Hi Getnamo,

Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

Got any idea on how to fix this?

[Carlosan]

Hi

On 4.22.0

Content > Modules > Rigging > Leap Mannequin Merged

Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

Thx in advance

[getnamo]

Hi Getnamo,

Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

Got any idea on how to fix this?

The general flow is to drag and drop the plugins folder from https://github.com/leapmotion/LeapUnreal/releases into your project of choice. An easy way to test this is to get the modules project https://github.com/leapmotion/LeapUn...dules/releases and drag and drop the matching plugin into it. When you launch you should have a bunch examples to play with out of the box.

If you wish to package you may need to add a basic empty c++ class to your project so that it detects your project plugin and includes it in the packaging process.

There is also a 3.0.1 version of the plugin available directly in the engine, just enable it and it should be available.

[getnamo]

Hi

On 4.22.0

Content > Modules > Rigging > Leap Mannequin Merged

Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

Thx in advance

Have you tried the latest release 0.3.7? https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7 there was a bug in some of the assets that didn't have their automap set on startup.

[Carlosan]

Yes, using latest release 0.3.7
If i move Right hand, i see on the display the left hand moving. Same with arms
Pic attached. Thx

[getnamo]

Yes, using latest release 0.3.7
If i move Right hand, i see on the display the left hand moving. Same with arms
Pic attached. Thx

Desktop mode? You should be using https://github.com/leapmotion/leapun...-example-actor which sets the optimization for desktop mode (and changes the expected orientation of the leap motion device)

Confirm you get good tracking by trying out the DesktopHelloWorld.umap in https://github.com/leapmotion/LeapUnrealModules

[Carlosan]

Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

Thx for your time.

[getnamo]

Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

Thx for your time.

repo uses git lfs, if you do not use git lfs, download a release (.7z) at https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7. If you clone without lfs you may end up with an empty directory.

[Carlosan]

Sorry if I waste your time and thanks for your patience.
I followed all your steps but I still get this when moving my right hand.

L to R
LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim

[getnamo]

Sorry if I waste your time and thanks for your patience.
I followed all your steps but I still get this when moving my right hand.

L to R
LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim

I think I see the problem. Currently it's setup to have a standard tracking where left = left etc, but you want a mirror mode where you right hand is the character's left hand facing you. In your particular example, if you rotated your characters 180 degrees they would line up with your tracked hands.

To do a mirror mode, you'd need to reflect the tracking data about a Z plane (math: http://mathworld.wolfram.com/Reflection.html but you can likely do vector subtraction from a fixed point for simpler math). There's nothing built in that supports it in e.g. bodystate, instead I'd recommend starting with tracking data from a leap component, reflecting that data and feeding it into a mesh. See the direct rigging examples on how to use the leap component based setup https://github.com/leapmotion/LeapUn...igging/LowPoly