Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    OSX support

    In OSX, I was able to get the plugin to show up in the list of plugins, but nothing works. Maybe telling is that, even after building, Plugins\LeapMotion\Binaries\Mac is empty.

    First, to get it to build for OSX:
    +Binaries\Mac\libLeap.dylib (copied from SDK)
    +Plugins\LeapMotion\ThirdParty\LeapSDK\Lib\Mac\libLeap.dylib (copied from SDK)
    modify Plugins\LeapMotion\Source\LeapMotion\LeapMotion.build.cs to load Mac\libLeap.dylib
    The project will then build, and in the editor Window->Plugins, under Installed>Input>Controller, there is Leap Motion Plugin v 0.8.2 and it is checked as enbaled.

    However, I can't seem to access it ...

    (1) ... using the provided content.
    Change the game mode to RenderToTexture.
    Change the Default Pawn Class to LeapRiggedCharacter.
    Try to launch in viewport, then get a warning:
    One or more blueprints has an unresolved compiler error, are you sure you want to Play in Editor?
    LeapDebugHand
    LeapDebugCharacter
    LeapBasicRiggedCharacter
    BasicCharacter_AnimBlueprint
    Clicking on yes gives me "BLUEPRINT COMPILER ERROR" in read text within the viewport.
    If I try to edit the LeapRiggedCharacter by double clicking on the blueprint in the content browser, then the editor crashes. This is the message I get in XCode:
    Code:
    [2014.12.17-17.56.00:057][475]LogRHI:Warning: OpenGL debug output extension not supported!
    [2014.12.17-17.56.05:856][672]LogRHI:Warning: OpenGL debug output extension not supported!
    [2014.12.17-17.56.06:059][672]LogRHI:Warning: OpenGL debug output extension not supported!
    Assertion failed: InSCSNode->ComponentTemplate != NULL
    (2) ... within a Character blueprint.
    I reverted back to the default Game Mode. Made a new Character Class blueprint. Click on Blueprint Props, scroll down to Interfaces, click on the Implemented Interfaces>Add drop down box and Leap is not there.

    (3) ... within the Engine Input settings. No Leap inputs can be found in any gamepad, keyboard, mouse, etc.

    Any ideas?

    By the way, I once read somewhere that you can modify PluginName.build.cs so that it copies the library into MyProject\Binaries\Platform. That way you don't need to provide the Binaries directory in your repository, then you'd be able to refactor your repository so it could simply be copied into the Plugins directory and maybe even be a submodule of another project. (Except for the optional content directory, but I think plugins can/will support content inclusion).

    Comment


      Originally posted by knack View Post
      I supposed that its the finger problem:
      https://www.youtube.com/watch?v=Ya_s3WLLAJI
      I found too a problem when move the HMD (last minute in video i think) the arms move. Your example proyect have same problem.
      Nice video, it does seems like the same problem, let me know if you find a fix. Unsure of the exact cause, but maybe it has to do with how yaw is set by the leap or the finger FK order; to be determined...

      Originally posted by knack View Post
      ...
      thanks for the plugin work. Still looking step by step how works i not understand few thinks of that blueprints. Ins't basic your example! :lol:
      You're welcome and yeah the rigged character is a little bit complicated because I'm preparing for the transition to input hardware agnostic rigging. For this reason there is a intermediate data structure (anim body) which holds and translates all the data into component space. This allows you to use the same anim body, but set by another hardware (e.g. hydra) and the rigging would remain valid.

      Note that thanks to the UE animation retargeting system you can copy animation blueprints (e.g. the basic one) to your own skeletal mesh if you've set up the skeleton bone remapping using the retarget manager.

      Originally posted by bullale View Post
      In OSX, I was able to get the plugin to show up in the list of plugins, but nothing works. Maybe telling is that, even after building, Plugins\LeapMotion\Binaries\Mac is empty.

      First, to get it to build for OSX:
      +Binaries\Mac\libLeap.dylib (copied from SDK)
      +Plugins\LeapMotion\ThirdParty\LeapSDK\Lib\Mac\libLeap.dylib (copied from SDK)
      modify Plugins\LeapMotion\Source\LeapMotion\LeapMotion.build.cs to load Mac\libLeap.dylib
      The project will then build, and in the editor Window->Plugins, under Installed>Input>Controller, there is Leap Motion Plugin v 0.8.2 and it is checked as enbaled.

      However, I can't seem to access it ...

      (1) ... using the provided content.
      Change the game mode to RenderToTexture.
      Change the Default Pawn Class to LeapRiggedCharacter.
      Try to launch in viewport, then get a warning:

      Clicking on yes gives me "BLUEPRINT COMPILER ERROR" in read text within the viewport.
      If I try to edit the LeapRiggedCharacter by double clicking on the blueprint in the content browser, then the editor crashes. This is the message I get in XCode:
      Code:
      [2014.12.17-17.56.00:057][475]LogRHI:Warning: OpenGL debug output extension not supported!
      [2014.12.17-17.56.05:856][672]LogRHI:Warning: OpenGL debug output extension not supported!
      [2014.12.17-17.56.06:059][672]LogRHI:Warning: OpenGL debug output extension not supported!
      Assertion failed: InSCSNode->ComponentTemplate != NULL
      (2) ... within a Character blueprint.
      I reverted back to the default Game Mode. Made a new Character Class blueprint. Click on Blueprint Props, scroll down to Interfaces, click on the Implemented Interfaces>Add drop down box and Leap is not there.

      (3) ... within the Engine Input settings. No Leap inputs can be found in any gamepad, keyboard, mouse, etc.

      Any ideas?

      By the way, I once read somewhere that you can modify PluginName.build.cs so that it copies the library into MyProject\Binaries\Platform. That way you don't need to provide the Binaries directory in your repository, then you'd be able to refactor your repository so it could simply be copied into the Plugins directory and maybe even be a submodule of another project. (Except for the optional content directory, but I think plugins can/will support content inclusion).
      Good investigation! Haven't had the opportunity to bind anything except windows so far, but I will be traveling with my old mac in the coming weeks so should have an opportunity to see how OSX integration would look like. Will take what you've found and poke at things to see what works and doesn't, I imagine the devil is in the details. We should try something much simpler for a start to see if a basic bind can be achieved and then move from there.

      Let me know if you can find a link to the PluginName.build.cs copy modification, this would be quite useful for all plugins with dependencies.
      Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

      Comment


        Originally posted by getnamo View Post
        Good investigation! Haven't had the opportunity to bind anything except windows so far, but I will be traveling with my old mac in the coming weeks so should have an opportunity to see how OSX integration would look like. Will take what you've found and poke at things to see what works and doesn't, I imagine the devil is in the details. We should try something much simpler for a start to see if a basic bind can be achieved and then move from there.
        I have a student working on writing a plugin for some of our scientific hardware and we are using your plugins as examples (in addition to actually using the Leap too). We're still learning programming for UE4, but I think a minimal plugin example that actually did something would be tremendously helpful, and much better than what's currently in the Wiki. For maximum community benefit this minimal plugin should probably not require hardware, but then it won't help us debug the OS X problem . I'm not at my office with my toys/research equipment every day but I'll take another look at getting a minimal Leap plugin working sometime next week.

        Originally posted by getnamo View Post
        Let me know if you can find a link to the PluginName.build.cs copy modification, this would be quite useful for all plugins with dependencies.
        It was a while ago. I guess I didn't remember it correctly. I had assumed it would be possible to code in some file-copy operations into PluginName.build.cs, but upon searching I realized the poster was running a custom script. See jyakku's response to the accepted answer here.

        Comment


          now i checking things about collision and taking objects (my head its overloading after try understand how works all this, rotations, etc and make it work ) but later next week i will try fix that. If i got something i write something.
          Last edited by knack; 12-18-2014, 01:41 PM.
          pd: excuse my english.

          lfw/paid modeling, painting, texturing.

          Comment


            Hi,
            I have dragged and dropped the binaries, plugins and content to my project root, then restarted UE4. When I launch the project I get this error message below. At this point I did not change any project/game settings, and have tried tested it just with the stater content but I get the same error. Disabling the plugin stops the error and the project runs fine.
            Click image for larger version

Name:	ph.jpg
Views:	1
Size:	224.8 KB
ID:	1064276
            I would be greatfull for any help.
            Thanks, Lewis

            Comment


              Originally posted by bullale View Post
              I have a student working on writing a plugin for some of our scientific hardware and we are using your plugins as examples (in addition to actually using the Leap too). We're still learning programming for UE4, but I think a minimal plugin example that actually did something would be tremendously helpful, and much better than what's currently in the Wiki. For maximum community benefit this minimal plugin should probably not require hardware, but then it won't help us debug the OS X problem . I'm not at my office with my toys/research equipment every day but I'll take another look at getting a minimal Leap plugin working sometime next week.
              There are two fully barebone plugin code examples that are included in each engine release. If you wan't to see a pretty barebone input bind plugin use the https://github.com/Ikarus76/UEJoystickPlugin as it only binds the windows direct input api to input mapping. The leap plugin is unfortunately to big in my opinion to properly encapsulate in the (FModulePlugin.cpp) module and I chose to set the classes directly to their UObject analogs instead, which works great because the leap SDK offloads that to another thread underneath.

              Originally posted by bullale View Post
              It was a while ago. I guess I didn't remember it correctly. I had assumed it would be possible to code in some file-copy operations into PluginName.build.cs, but upon searching I realized the poster was running a custom script. See jyakku's response to the accepted answer here.
              This is what the plugins currently already use to link to libs and dlls, it doesn't copy anything in the packaging process unfortunately. If you link to a lib it expects the .dll to be in the project or engine binaries folder not the plugin one, the only way around it so far is to do a direct dll bind which is not recommended.

              Originally posted by lewisq View Post
              Hi,
              I have dragged and dropped the binaries, plugins and content to my project root, then restarted UE4. When I launch the project I get this error message below. At this point I did not change any project/game settings, and have tried tested it just with the stater content but I get the same error. Disabling the plugin stops the error and the project runs fine.
              ...
              I would be greatfull for any help.
              Thanks, Lewis
              Launch is currently not supported, use Play In Editor and Play Standalone for testing. You can also package and ship following the shipping instructions on the github repo.

              The reason for this is that currently there is no way to hook into the packaging process to move the required files to the right directory for the project to see the dependencies automatically. Waiting on epic to add necessary fixes before launch will work properly.
              Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

              Comment


                First off, great work on the plug in, this very well done . Unfortunately, I'm running into some problems. My end goal is to use a tool (e.g. pencil) to move a crosshair around. Step 1 is to get a non-tick event to work (like detect hands, etc.), and the images below are my attempt to use the raw image event. The "Hello" statement fires, but "It works?" does not.

                Click image for larger version

Name:	AtBeginEvent.PNG
Views:	1
Size:	75.3 KB
ID:	1064319Click image for larger version

Name:	Doesn'tPrint.PNG
Views:	1
Size:	70.2 KB
ID:	1064320

                Also, I tried to get a tool detected using tick and then used tool->count->print string(count) and it always says the count is zero. I've also tried getting the direction of the tool, but it always at 0,0,0. Image below is one of my many attempts to find a working solution (tick is not connected in the image below). Any suggestions?

                Click image for larger version

Name:	ToolAttempt.PNG
Views:	1
Size:	61.4 KB
ID:	1064321

                Comment


                  i read the latest leap motion sdk have API changes. beware at update. I not update.
                  pd: excuse my english.

                  lfw/paid modeling, painting, texturing.

                  Comment


                    Knack, good call I did not notice that there was a big change from 2.2.0 to 2.2.1. So, LEAP SDK 2.2.1 is not supported? Uh oh, well I don't have 2.2.0 or lower. I just tried google trying to find the previous version but no luck. I doubt I could edit my version of the leap plugin for the new bindings without messing up somewhere, is there somewhere that I can find leap sdk 2.2.0?

                    Version 2.2.1

                    Added APIs for all language bindings (except JavaScript):
                    Controller.images()
                    Listener.onImages()
                    Image.sequenceID()
                    Added LeapFrame serialize and deserialize functions for Objective-C language binding.

                    Comment


                      today my leap upgrade without have checked automatic upgrades.

                      but works. I read something about API changes on images only (i read fast), check the changelog.
                      pd: excuse my english.

                      lfw/paid modeling, painting, texturing.

                      Comment


                        https://developer.leapmotion.com/downloads
                        pd: excuse my english.

                        lfw/paid modeling, painting, texturing.

                        Comment


                          i can't fix my fingers problem.

                          (i hope i can explain with my bad english)

                          I notice my lowerarm using left_elbow_transform like your template getnamo not twist/roll, i add the ROLL from my hand to the lowerarm and seems better. (+hand roll - 180 for left)

                          The same problem with finger, not roll when roll the hand, i don't now where are the problem, if i move the finger after roll the hand, then the finger roll right. Its hard notice with the bluecharacter but i think your template getnamo have the same fingers problems.

                          Like with lowerarm i check adding the roll value to my index finger (+hand roll -90) and seems work fine if i roll the hand, finger roll, the problem now its when i move the finger after adding the roll value of hand, now i have incorrect roll values in my index finger.

                          I have a headhache with all this rotations, and try understand. :lol:

                          I made a video (disregar skin weight problems)
                          https://www.youtube.com/watch?v=g3C6KTS5I78

                          not show in that video still not understand why if i move the head the elbows move, that must be a driver problem.

                          edit: if i place my hand in front of my face and move the head with the hand following keeping the position in front of my faces, the finger rotate. That must be the problem clearly.

                          here another test
                          https://www.youtube.com/watch?v=c5HgHQwYbvY

                          and this another video
                          (index finger)
                          https://www.youtube.com/watch?v=s_29...ature=youtu.be
                          Last edited by knack; 12-27-2014, 06:59 AM.
                          pd: excuse my english.

                          lfw/paid modeling, painting, texturing.

                          Comment


                            Hello there! First of all, thanks for this great work that provides us this fantastic plugin!

                            I have an issue to report: whenever I use the HAND function (OnFingerMoved->LeapPointable->Hand) - to get the hand reference and determine its type - UE4 editor crashes when I exit the game (launched in editor window, standalone window or separate thread - doesn't matter).

                            When I remove the HAND node, everything behaves properly.

                            In the imgur link I've attached screenshots of the node graph & crash report - although the report doesn't say anything helpful

                            http://imgur.com/a/D6YiL

                            Hope it's helpful!

                            Cheers
                            Mikołaj Bazaczek / VR Lead / Herzog & de Meuron
                            m.bazaczek@herzogdemeuron.com / mbazaczek@gmail.com
                            3dsmax / UE4 / VRay / Corona / Rhino
                            js / python / web / php

                            Comment


                              i haven't that problem in standalone but in editor window UE4 crash.


                              i think i discovery where is the problem with my fingers

                              i don't known how to solve. :|

                              Leap motion seems use negative rotation numbers when the palm/fingers its fancing the camera and positive when no.

                              then in my video example
                              https://www.youtube.com/watch?v=c5HgHQwYbvY

                              when finger rotation reach 180 the next value its -180 and the finger change the orientation. The same thing when passing from -180 to 180.

                              i think the solution its have 0 -360º a 0 -360º depending if finger are facing the camera or no.

                              any idea? thanks.
                              Last edited by knack; 12-28-2014, 03:29 PM.
                              pd: excuse my english.

                              lfw/paid modeling, painting, texturing.

                              Comment


                                Originally posted by aerloth View Post
                                Hello there! First of all, thanks for this great work that provides us this fantastic plugin!

                                I have an issue to report: whenever I use the HAND function (OnFingerMoved->LeapPointable->Hand) - to get the hand reference and determine its type - UE4 editor crashes when I exit the game (launched in editor window, standalone window or separate thread - doesn't matter).

                                When I remove the HAND node, everything behaves properly.

                                In the imgur link I've attached screenshots of the node graph & crash report - although the report doesn't say anything helpful

                                http://imgur.com/a/D6YiL

                                Hope it's helpful!

                                Cheers
                                Try making sure your leap motion settings (right click in taskbar) are correct, and see if that fixes the problem. If not, hmmm not sure.
                                Attached Files

                                Comment

                                Working...
                                X