Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by Carlosan View Post
    Hi

    On 4.22.0

    Content > Modules > Rigging > Leap Mannequin Merged

    Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

    Thx in advance
    Have you tried the latest release 0.3.7? https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7 there was a bug in some of the assets that didn't have their automap set on startup.

    Leave a comment:


  • replied
    Originally posted by AydinMAD View Post
    Hi Getnamo,

    Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

    When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
    I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

    I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

    Got any idea on how to fix this?
    The general flow is to drag and drop the plugins folder from https://github.com/leapmotion/LeapUnreal/releases into your project of choice. An easy way to test this is to get the modules project https://github.com/leapmotion/LeapUn...dules/releases and drag and drop the matching plugin into it. When you launch you should have a bunch examples to play with out of the box.

    If you wish to package you may need to add a basic empty c++ class to your project so that it detects your project plugin and includes it in the packaging process.

    There is also a 3.0.1 version of the plugin available directly in the engine, just enable it and it should be available.

    Leave a comment:


  • replied
    Hi

    On 4.22.0

    Content > Modules > Rigging > Leap Mannequin Merged

    Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

    Thx in advance
    Last edited by Carlosan; 04-21-2019, 10:56 AM.

    Leave a comment:


  • replied
    Hi Getnamo,

    Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

    When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
    I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

    I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

    Got any idea on how to fix this?

    Leave a comment:


  • replied
    Originally posted by NickieNickie
    Hey Getnamo, I was wondering if it would be possible to retarget the leap motion hands to a full body skeleton mesh's hands? I would like to use leap motion for 3D animation,

    I found this article: https://www.searis.net/assets/download/WilsonEtAl.pdf

    It looks like it is possible, but I don't know if this is too advanced for me yet. I guess i will order one now since the alternative is spending over $1000 for gloves.

    Thank you.
    There is an example of this kind of rigging for the unreal mannequin, found here: https://github.com/leapmotion/LeapUnrealmodules#rigging

    Leave a comment:


  • replied
    Originally posted by ibPrf View Post

    I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.
    It's already in the engine (it has finally been updated to the modern architecture!), remove the project plugin and enable the engine version. There will be a project version release available within a week of when 4.22 releases.

    Leave a comment:


  • replied
    Originally posted by Le H Ngo
    Hi Guys,

    Trying to packaga the demo scene VRPickupdrop but I keep getting the error below in the screenshot?

    ATHelper: Packaging (Lumin): mabu: error: D:/Unreal/ML_PL/Intermediate/Lumin/Mabu\manifest.xml: missing <application ml:min_api_level=...> attribute

    Anyone knows how to fix it please?

    Tried to edit xml file but with ml:min_api_level="1" but it won't save down

    Many Thanks
    Lumin platform is Magic Leap, not Leap Motion. Wrong plugin!

    Leave a comment:


  • replied
    Originally posted by getnamo View Post

    The current plugin found at https://github.com/leapmotion/leapunreal should work both as project plugin and engine plugin as it detects it's environment. The general way to rebuild the plugin is to include it in e.g. a ue4 C++ project under plugins and regenerating your .sln then recompiling, the plugin will recompile with it.

    The same process is used for engine plugins except that you replace the plugin found under e.g.
    Code:
    UE_4.21/Engine/Plugins/Runtime/LeapMotion
    and you copy your third party includes to
    Code:
    UE_4.21/Engine/Source/ThirdParty/Leap
    and dlls to
    Code:
    UE_4.21/Engine/Binaries/ThirdParty/LeapMotion
    I'm hoping we will have an updated leap motion plugin for 4.22 which should eliminate the engine plugin workaround, but can give no guarantees at this time.
    I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.

    Leave a comment:


  • replied
    Originally posted by lumbabumba View Post
    Hi,

    ty for your plugin. I am trying to get the camera image of the LeapMotion Controller to do some processing with OpenCV.
    I have added LeapMotion to my modules and I can compile.



    But I found this here...

    Is there any way to get images or frames to use OpenCV on it?

    Ty, sry for bad english.
    Answered here: https://github.com/leapmotion/LeapUnreal/issues/41. Use SetLeapPolicy to enable raw image streaming, then subscribe to the OnImageEvent on your leap component.

    Leave a comment:


  • replied

    Hi,

    ty for your plugin. I am trying to get the camera image of the LeapMotion Controller to do some processing with OpenCV.
    I have added LeapMotion to my modules and I can compile.



    But I found this here
    Code:
     
    /** Enable or disable image streaming from the device*/
    //UPROPERTY(BlueprintReadWrite, Category = "Leap Options") //disabled for now, current version doesn't actually support image streaming
    bool bEnableImageStreaming;
    Is there any way to get images or frames to use OpenCV on it?

    Ty, sry for bad english.
    Last edited by lumbabumba; 03-20-2019, 10:51 AM.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post

    Check out https://github.com/leapmotion/leapunrealmodules#rigging for rigging examples (download project, drag and drop matching unreal engine plugin and open the CustomRigHello.umap to see various assets rigged)
    Thanks! Sorry, missed that before - very handy.

    Afraid it looks like the flipping still hasn't gone away totally - for a while it seems really solid and then sometimes it comes back and sticks around. This happens both in UE and the visualizer, as seen here:

    https://vimeo.com/319510005/40f4932c07

    Are there any things that could lead to this do you know? Should I try getting another sensor or might it be something to do with the conditions it's used in? It seems similar to the issue here:
    https://forums.leapmotion.com/t/hand...cognition/7699

    If there were a way to let it know that it would only ever be looking at a right hand then this would presumably solve it - but I don't know if there is?

    Leave a comment:


  • replied
    Originally posted by KimLaughton View Post
    Actually I already turned off the deformation, so I guess it's not that. Not quite sure why it's so far from where it ought to be because the hand rig looks fairly conventional to me. There were some UE hands/arms for Leap Motion around before, but they don't seem to be included in the current plugin. Are they still available and is the rigging still suitable for the latest version of the software?

    Setting it to desktop mode seems to have solved the flipping thanks! Also great to have the option for more smooth input, it was a bit jittery before.


    Check out https://github.com/leapmotion/leapunrealmodules#rigging for rigging examples (download project, drag and drop matching unreal engine plugin and open the CustomRigHello.umap to see various assets rigged)

    Leave a comment:


  • replied
    Actually I already turned off the deformation, so I guess it's not that. Not quite sure why it's so far from where it ought to be because the hand rig looks fairly conventional to me. There were some UE hands/arms for Leap Motion around before, but they don't seem to be included in the current plugin. Are they still available and is the rigging still suitable for the latest version of the software?

    Setting it to desktop mode seems to have solved the flipping thanks! Also great to have the option for more smooth input, it was a bit jittery before.

    Originally posted by getnamo View Post

    It looks like your hand is deforming to fit the tracked hand data using Should Deform Mesh option found here (https://github.com/leapmotion/leapunreal#deformation)


    For this to work your model should have it's bones placed/closely weighted as the low poly hand does so that the hand correctly stretches to fit the user's real hand data. If you don't mind some tip misalignment or your model doesn't support deformation (e.g. ue4 mannequin model) then untick this option and it will apply rotations only to the hand which works far more universally.


    For hand chirality, are you using this attached to a VR headset or is it e.g. on a table facing up? If it's the latter, you'll need to call SetLeapMode (https://github.com/leapmotion/leapunreal#set-leap-mode) in e.g. your begin play and set it to LEAP_MODE_DESKTOP this should ensure the tracking software expects the correct chirality based on it's orientation. The example LeapDesktopActor has this call, open the blueprint to see where the calls are done.

    Leave a comment:


  • replied
    Originally posted by KimLaughton View Post

    Thanks so much for this info! I've fixed the thumb - I'm not sure what was causing it, but restarted from scratch and it's fine now. I have the right hand automap target and it's now mostly working fine but everything is a little off.

    vid here: https://gyazo.com/dc72381e293d8c17aada694da4d3ca8f

    At the moment it feels like the proximal bones aren't moving at all and all the others look v strange (maybe because of this?)

    Click image for larger version  Name:	handbones.jpg Views:	1 Size:	256.8 KB ID:	1586330

    The mapping looks ok to me. Maybe I should give up with this model and use something that's based on the epic skeleton - is that likely to give better results?

    As far as the flipping is concerned it seems to be happening in the Orion software - sometimes it's fine and completely solid, other times it there is flipping I guess as you say to due to the chirality detection. I'm not sure quite why it occurs, the environment and location of the sensor is always pretty much the same and the lens is clean. vid here https://gyazo.com/6572b1d4d08805fd898de4646ed31f9a
    It looks like your hand is deforming to fit the tracked hand data using Should Deform Mesh option found here (https://github.com/leapmotion/leapunreal#deformation)


    For this to work your model should have it's bones placed/closely weighted as the low poly hand does so that the hand correctly stretches to fit the user's real hand data. If you don't mind some tip misalignment or your model doesn't support deformation (e.g. ue4 mannequin model) then untick this option and it will apply rotations only to the hand which works far more universally.


    For hand chirality, are you using this attached to a VR headset or is it e.g. on a table facing up? If it's the latter, you'll need to call SetLeapMode (https://github.com/leapmotion/leapunreal#set-leap-mode) in e.g. your begin play and set it to LEAP_MODE_DESKTOP this should ensure the tracking software expects the correct chirality based on it's orientation. The example LeapDesktopActor has this call, open the blueprint to see where the calls are done.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post

    Inside your anim instance class defaults, Ensure automap target is HAND RIGHT for your case. Also below that section is a structure called mapped bone list open it up you can see Pre-base rotation and offset transform, these will apply to the entire mapped bone list. Pre-base rotation will apply rotation before your values (akin to changing the basis of your input) where as offset transform applies after your input has been fed. If you're moving your hand/fingers and they bend in a wrong axis e.g. they yaw when they should pitch, you will need to adjust the pre-base rotation, typically in some combination of 90 degrees. These will typically be different for each hand if your mesh is rigged like animators typically do with positive rotation closing your fist. In the low poly hands it uses a pre base of -90 for left and 90,0 180 for right.

    That said it looks like your thumb might be in a different space than the rest of your hand and you may need to remove it from the auto mapping list and feed the rigging directly, or go back and adjust your model in your software. If it's just a fixed offset difference however, you can just add a standard Transform (Modify) Bone node and adjust the offset there after the Modify Mapped Bones node.

    If the auto-rigging is giving you a problem, checkout the discussion here for basic details on direct rigging:
    https://github.com/leapmotion/LeapUnreal/issues/35


    Regarding the flipping, it might be occurring due to hand chirality detection, might need a video of the issue to know more.
    Thanks so much for this info! I've fixed the thumb - I'm not sure what was causing it, but restarted from scratch and it's fine now. I have the right hand automap target and it's now mostly working fine but everything is a little off.

    vid here: https://gyazo.com/dc72381e293d8c17aada694da4d3ca8f

    At the moment it feels like the proximal bones aren't moving at all and all the others look v strange (maybe because of this?)

    Click image for larger version  Name:	handbones.jpg Views:	1 Size:	256.8 KB ID:	1586330

    The mapping looks ok to me. Maybe I should give up with this model and use something that's based on the epic skeleton - is that likely to give better results?

    As far as the flipping is concerned it seems to be happening in the Orion software - sometimes it's fine and completely solid, other times it there is flipping I guess as you say to due to the chirality detection. I'm not sure quite why it occurs, the environment and location of the sensor is always pretty much the same and the lens is clean. vid here https://gyazo.com/6572b1d4d08805fd898de4646ed31f9a

    Leave a comment:

Working...
X