Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Originally posted by lumbabumba View Post
    Hi,

    ty for your plugin. I am trying to get the camera image of the LeapMotion Controller to do some processing with OpenCV.
    I have added LeapMotion to my modules and I can compile.



    But I found this here...

    Is there any way to get images or frames to use OpenCV on it?

    Ty, sry for bad english.
    Answered here: https://github.com/leapmotion/LeapUnreal/issues/41. Use SetLeapPolicy to enable raw image streaming, then subscribe to the OnImageEvent on your leap component.
    Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

    Comment


      Originally posted by getnamo View Post

      The current plugin found at https://github.com/leapmotion/leapunreal should work both as project plugin and engine plugin as it detects it's environment. The general way to rebuild the plugin is to include it in e.g. a ue4 C++ project under plugins and regenerating your .sln then recompiling, the plugin will recompile with it.

      The same process is used for engine plugins except that you replace the plugin found under e.g.
      Code:
      UE_4.21/Engine/Plugins/Runtime/LeapMotion
      and you copy your third party includes to
      Code:
      UE_4.21/Engine/Source/ThirdParty/Leap
      and dlls to
      Code:
      UE_4.21/Engine/Binaries/ThirdParty/LeapMotion
      I'm hoping we will have an updated leap motion plugin for 4.22 which should eliminate the engine plugin workaround, but can give no guarantees at this time.
      I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.

      Comment


        Originally posted by Le H Ngo
        Hi Guys,

        Trying to packaga the demo scene VRPickupdrop but I keep getting the error below in the screenshot?

        ATHelper: Packaging (Lumin): mabu: error: D:/Unreal/ML_PL/Intermediate/Lumin/Mabu\manifest.xml: missing <application ml:min_api_level=...> attribute

        Anyone knows how to fix it please?

        Tried to edit xml file but with ml:min_api_level="1" but it won't save down

        Many Thanks
        Lumin platform is Magic Leap, not Leap Motion. Wrong plugin!
        Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

        Comment


          Originally posted by ibPrf View Post

          I hope that you are not getting overwhelmed with this question but do you have any idea on when a 4.22 plugin might be available? Even a work around would be great. My team is eager to move forward with ray-tracing but we would really hate to have to go back to using controllers.
          It's already in the engine (it has finally been updated to the modern architecture!), remove the project plugin and enable the engine version. There will be a project version release available within a week of when 4.22 releases.
          Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

          Comment


            Originally posted by NickieNickie
            Hey Getnamo, I was wondering if it would be possible to retarget the leap motion hands to a full body skeleton mesh's hands? I would like to use leap motion for 3D animation,

            I found this article: https://www.searis.net/assets/download/WilsonEtAl.pdf

            It looks like it is possible, but I don't know if this is too advanced for me yet. I guess i will order one now since the alternative is spending over $1000 for gloves.

            Thank you.
            There is an example of this kind of rigging for the unreal mannequin, found here: https://github.com/leapmotion/LeapUnrealmodules#rigging
            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

            Comment


              Hi Getnamo,

              Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

              When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
              I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

              I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

              Got any idea on how to fix this?

              Comment


                Hi

                On 4.22.0

                Content > Modules > Rigging > Leap Mannequin Merged

                Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

                Thx in advance
                Last edited by Carlosan; 04-21-2019, 10:56 AM.

                Comment


                  Originally posted by AydinMAD View Post
                  Hi Getnamo,

                  Thank you for making this plugin. I'm a bit new to Unreal and I'm having an issues with the leapmotion plugin. First of all in the Editor I have no issues with getting the hand data in, but none of the plugin content is visible. Changing the view options to show plugin content doesn't change anything.

                  When I put the plugin in the engine installation folder Plugins directory and try to package my project the plugin doesn't work in the executable. When I put the Plugins folder in the project folder and remove it from the engine installation folder it also won't work. The output log doesn't show any useful information and the package gets compiled successfully.
                  I also don't see the LeapMotion plugin in the plugins list when I place the Plugins folder in my project root.

                  I'm using Unreal Engine 4.22 with the LeapMotion-v3.3.1-UE4.22.7z version. I think the plugin might not be working with UE 4.22.

                  Got any idea on how to fix this?
                  The general flow is to drag and drop the plugins folder from https://github.com/leapmotion/LeapUnreal/releases into your project of choice. An easy way to test this is to get the modules project https://github.com/leapmotion/LeapUn...dules/releases and drag and drop the matching plugin into it. When you launch you should have a bunch examples to play with out of the box.

                  If you wish to package you may need to add a basic empty c++ class to your project so that it detects your project plugin and includes it in the packaging process.

                  There is also a 3.0.1 version of the plugin available directly in the engine, just enable it and it should be available.
                  Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                  Comment


                    Originally posted by Carlosan View Post
                    Hi

                    On 4.22.0

                    Content > Modules > Rigging > Leap Mannequin Merged

                    Movement is captured ok but arms/hands axis are inverted. Any tip to solve it ?

                    Thx in advance
                    Have you tried the latest release 0.3.7? https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7 there was a bug in some of the assets that didn't have their automap set on startup.
                    Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                    Comment


                      Yes, using latest release 0.3.7
                      If i move Right hand, i see on the display the left hand moving. Same with arms
                      Pic attached. Thx
                      Attached Files

                      Comment


                        Originally posted by Carlosan View Post
                        Yes, using latest release 0.3.7
                        If i move Right hand, i see on the display the left hand moving. Same with arms
                        Pic attached. Thx
                        Desktop mode? You should be using https://github.com/leapmotion/leapun...-example-actor which sets the optimization for desktop mode (and changes the expected orientation of the leap motion device)

                        Confirm you get good tracking by trying out the DesktopHelloWorld.umap in https://github.com/leapmotion/LeapUnrealModules
                        Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                        Comment


                          Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

                          But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

                          Thx for your time.

                          Comment


                            Originally posted by Carlosan View Post
                            Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

                            But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

                            Thx for your time.
                            repo uses git lfs, if you do not use git lfs, download a release (.7z) at https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7. If you clone without lfs you may end up with an empty directory.
                            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                            Comment


                              Sorry if I waste your time and thanks for your patience.
                              I followed all your steps but I still get this when moving my right hand.

                              L to R
                              LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim

                              Attached Files

                              Comment


                                Originally posted by Carlosan View Post
                                Sorry if I waste your time and thanks for your patience.
                                I followed all your steps but I still get this when moving my right hand.

                                L to R
                                LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim
                                I think I see the problem. Currently it's setup to have a standard tracking where left = left etc, but you want a mirror mode where you right hand is the character's left hand facing you. In your particular example, if you rotated your characters 180 degrees they would line up with your tracked hands.

                                To do a mirror mode, you'd need to reflect the tracking data about a Z plane (math: http://mathworld.wolfram.com/Reflection.html but you can likely do vector subtraction from a fixed point for simpler math). There's nothing built in that supports it in e.g. bodystate, instead I'd recommend starting with tracking data from a leap component, reflecting that data and feeding it into a mesh. See the direct rigging examples on how to use the leap component based setup https://github.com/leapmotion/LeapUn...igging/LowPoly
                                Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                                Comment

                                Working...
                                X