User Tag List

Page 1 of 14 12311 ... LastLast
Results 1 to 40 of 526

Thread: [Plugin] Leap Motion - Event Driven

  1. #1
    1

    [Plugin] Leap Motion - Event Driven

    An event-driven Leap Motion plugin.

    Forked MarcW's plugin to update to the latest Leap SDK and add support for event driven architecture. Since the architecture differs, this new thread is made pertaining to the fork. Expect to see some of these changes make it back to my other plugins (interfaces rock). Full source is provided at github, if you feel you can help, feel free to extend and contribute. Let me know if any bugs crop up!

    NB: Currently only supports Windows 64 & 32 and Mac OSX Platforms.

    Want to try Orion? just update your runtime and you'll get the higher accuracy tracking. NB: The current plugin has more latency than the unity version for now.

    Download
    Latest Stable (4.11 +) - It's already in the engine, just enable and restart!
    Latest Dev (4.11) - For in-between engine fixes and new feature tests

    Older
    0.9.9 for UE4.10 / Leap SDK 2.3.1 - Mac Binaries synced to 4.10
    0.9.7 for UE4.9 / Leap 2.3.1 - (Mac UE4.7)
    0.9.5 for UE4.8 / Leap 2.2.6 - (Mac UE4.7)
    0.9.3 for UE4.7 / Leap 2.2.4
    0.8.3 for UE4.6 / Leap 2.2.1
    0.8.1 for UE4.5 / Leap 2.2
    0.6.1 for UE4.4 / Leap 2.1.5

    Resources
    Github - Main Plugin Source and Documentation
    Leap Motion API documentation - Answers the question: How do I do X?
    Leap Community UE4 Plugin Thread

    How to Use

    1. Install plugin by Downloading and Draging Binaries and Plugins into your project root. (Optional content automatically included since 0.9, delete if you're not using it and you wish to save space)



    2. Open your project and confirm that the plugin is enabled (Window->Plugins, should be under Installed category).

    3A. Easy Option (Since 0.8):

    3A.1 Set your Character to LeapRiggedCharacter, Set your PlayerController to VRPlayerController (if using hmd). See Epic's game mode documentation if you're confused on how to change these.



    3A.2 Play to try it out! Use VRPreview for HMD mode (Do not use launch)



    3A.3 - Optional (since 0.9.3) If you want collision enabled for you fingers, simply change your collision preset to PhysicsActor. If you don't know what that is, use the convenience LeapCollisionCharacter instead of LeapRiggedCharacter. This allows you to interact with movable actors with physics enabled.



    3A.4 - Optional
    Pass-through is supported from 0.9, useful if you wish to blend real-world for AR purposes or simply wish to see where you type in vr.
    Simply select LeapPassthroughCharacter as your pawn and keep the controller as VRController


    3A.4 Play to try out AR/VR transition with a simple gesture


    Convenience content relation diagram


    This setup gives you a wide variety of points to sub-class or modify any convenience blueprint where you wish to change functionality.

    3B. Custom Option: Select the blueprint where you want to receive Leap Events, select Class Settings



    3B.1 Under Details find Interfaces and Add LeapEventInterface to your blueprint.
    3B.2 Add a Leap Controller component by clicking Add Component



    3B.3. (Optional) Right click event graph and from the LeapController set OptimizeForHMD to true if using leap mounted to hmd rather than default on table facing up, keep auto rotate and auto shift if you want the plugin to automatically shift the leap positions to account for HMD movement.

    Debug Hands

    The plugin is now ready to use. Right click on Event Graph and type 'Leap Events' to show all the events you receive, implement the ones you wish to respond to. If you saved your LeapController pointer as a variable, you can also poll any of the latest 60 frames using Leap API. Below is an example of how to get 'Debug Hands' via events.



    With Draw Debug Sphere At Actor defined as


    This will give you the following in the blank template with starter content.



    Please note that this is for the default mode (on table facing up) and you may need to adjust your leap-actor offset. For hmd mode with auto shift and rotate ticked, the values should be passed straight through (with added actor location if you're using debug spheres) as they are already automatically compensated due to the plugin knowing the position of the rift.

    Gestures
    Plugin supports Leap gesture api. You must first enable the gesture you wish to recognize and then listen for your event. Example gesture recognizer setup



    then you can do sub-gesture specific tasks either by casting your gesture to your sub-gesture or subscribing to the specific sub-gesture event. Example of receiving circle gesture event



    Input Mapping
    Useful for simple gameplay mechanics or input parts which behave like joystick axis or buttons. NB: You need to have a Leap Component present in any one of your active blueprints for input mapping events to emit, same as any other blueprint use.

    E.g. adding one input mapping key and 2 axis to the Rolling Template, allows you to roll the ball by the pitch and roll of your hand and jump by closing your fist. Simple fun!



    Changelist

    2.0.0
    -Integrated directly in 4.11 engine
    -Added Image Hands
    -Added Component Hands
    -Thorough code commenting
    -Automatic packaged support
    -Added convenience LeapFloatingHandsCharacter

    0.9.9
    -Updated Mac binary for 4.10

    0.9.8
    -Updated binaries to UE4.10

    0.9.7
    -Fixed packaging for 4.9.2
    -Added EnableBackgroundTracking toggle on LeapController allowing you to
    track when the window isn't focused
    -Updated Echo hands to hide when not being tracked, idle
    animations for Echo hands have been removed. Use
    LeapEchoHandsRiggedCharacter for a rigged mesh hand without attached
    body, use the collision variant for physics enabled version.

    0.9.6
    -Binaries synced with UE 4.9
    -Leap SDK synced with 2.3.1
    -Added Echo convenience hands (rigged floating hands)

    0.9.5
    -Updated binaries to UE4.8

    0.9.4
    -Updated Leap SDK to 2.2.6

    0.9.3
    -Added LeapCollisionCharacter which adds physics collision enabled by
    default. This can be enabled for other setups by using the PhysicsActor
    collision preset.
    -Updated PhysicsAsset used for collision
    -Removed old content
    -Fixed SphereRadius property link found on LeapHand

    0.9.2
    -Mac Binaries compiled for UE 4.7
    -Leap SDK, libs and dlls updated for SDK 2.2.4

    0.9.1
    -Cleaned up inheritence structure, LeapAnimBodyActor renamed to
    LeapAnimBodyConnector. Added Passthrough sub-class
    -LeapPassthroughCharacter now contains the subclass. Rigged characters
    contain no passthrough code
    -Fixed elbow rotation bug

    0.9
    -Added properly HMD warped image passthrough support through both
    materials and a convenience character (LeapPassthrougCharacter)
    -128bit texture support resolved for Distortion textures. Use Distortion
    for proper float data, use DistortioUE to visualize the texture for
    debug purposes.
    -Optional content moved into plugin, delete if you don't use it and wish
    to save space
    -Rigging update to be more loosely coupled. Character anim blueprint
    only depends on AnimBody obtained through a simple and safe
    AnimBodyInterface
    -Rigged character finger rotation bug fixed
    -Updated leap bind to 2.2.3, much improved tracking gained.
    -Added R->Reset position to rigged characters, change key bind to
    reflect own bindings

    0.8.3
    -Now supports Mac OSX
    -Fixed Finger->Hand crash on exit
    -Update to Leap SDK 2.2.1

    0.8.2
    -Now supports UE 4.6

    0.8.1
    -Fixed Memory leaks which would cause crash around 60second mark (GC
    interval) for list-based objects using extended and append
    -Added appendTools and appendFingers to pointableList
    -LeapFrame got its API updated to include all of the missing functions,
    should now reflect the leap api nearly completely (except 64bit values
    in blueprint that are not supported)
    -Added log entry with plugin version at game start for easier debugging

    0.8.0
    -Basic Rigged leap character added. Works like the LeapDebugCharacter,
    change your player controller to VRPlayercontroller and pawn to
    LeapRiggedCharacter
    -Updated bind to Leap SDK 2.2.0
    -Ensured consistency by prepending leap to all classes, this will likely
    break compatibility with old blueprints.
    -Fixed dynamic object allocation architecture
    -Added basic AnimBody, Hand, Finger, and Bone for convenient animation
    state and basis conversion.

    0.7.11
    -You can now check which finger you have by dragging out the Type
    property from Finger.

    0.7.10
    -Added a convenience Leap Debug Hand actor, allowing for easy display of
    debug hands
    -Added a convenience Leap Debug Character which is set up to show debug
    hands in both HMD and Default mode. Simply toggle your hmd (alt+enter)
    to test HMD mode.

    0.7.9
    -added Input Mapping support for pinch grab and palm orientation for
    each hand. See documentation for details
    -added PalmOrientation convenience function to Leap Hand. Origin is
    defined as palm flat facing down, fingers pointing away.

    0.7.8
    -Update to Leap SDK 2.1.6. Includes changes from leap
    (https://developer.leapmotion.com/doc...#version-2-1-6)

    0.7.7
    -If you tick auto-shift on hmd, this will automatically add the
    eye->leapmount offset (8cm default value). Your vr hands should now be
    completely 1:1
    -Added Hand Type enum to Hand and relevant gestures. You can now easily
    switch the hand type instead of checking isLeft/Right to reduce bp
    clutter.
    -Added Leap Plugin Direction (Cardinal Direction) to gestures which
    emit directions (Screen Tap, Key Tap, and Swipe). This is a convenience
    property which will allow you to easily switch the property to get basic
    swipe left/right/up/down/towards/away detection. If you need a more
    precise direction, use the direction vector.

    0.7.6
    -Added Raw Image events which give much cleaner support and memory
    efficient toggling of streaming images from the device.
    -Multiple components are now properly supported. Reason for error was
    mismanaged GC of on-demand UObject subclasses. Plugin will still emit
    multi-component warning for performance reasons (redundancy/duplication).
    -All logging switched to LeapPluginLog sub-category

    0.7.5
    -At the moment multiple components cause a crash after 40-60sec of
    detecting a hand/finger, until this error is resolved, the plugin will
    output a warning to your logs if you add more than one total leap
    component

    0.7.4
    -Normalized vectors such as palm normal were not properly adjusted for
    HMD view direction, this is has now been fixed.

    0.7.3
    -palm normal scaling fixed.

    0.7.2
    -Circle gesture radius now in UE scale
    -Swipe Gesture speed now in UE scale

    0.7.1
    -Hidden Leap header and private implementation from UClasses
    -Fixed image stability, will now correctly resize when requested.
    -R8 texture no longer causes memory leaks
    -Added Gesture sub-classes and corresponding events (e.g. Swipe Gesture
    Detected). Will allow for easy determination of direction and other
    properties of gestures. You can also cast general gesture detection to
    the correct sub-class gesture.
    -Added own category of logging, pay attention to log output to help you
    with debugging errors.

    0.7
    -Added full support for Image API! see post or github readme for an example use case.
    -Fixed a lot of memory leaks, should have no more memory leaks and purr like a kitten.

    0.6.4
    -Fixed lack of categorization caused by previous updates in some of the blueprints classes
    -Cleaned up a lot of the blueprint functions, turning all the relevant
    ones into properties. This will reduce the number of execution pins.
    -Things which return a new object generally remain a function for
    potential performance reasons

    0.6.3
    -Changed plugin to runtime to support shipping builds
    -See the github readme for shipping/packaging instructions.

    0.6.2
    -Updated to UE4.5
    -Removed the need to SetInterfaceDelegate! Just adding the Component and
    adding the LeapEventInterface is enough.
    -Added Image policy flag support, actual image handling is still not
    working at this time. You can query about image dimension and other
    stats, just not the raw image.

    0.6.1
    -OptimizeForHMD now contains 2 more booleans, one for auto rotation
    adjustment and one for auto position adjustment. Setting all three to
    enabled will make your reported leap positions automatically adjust for
    your head position and look direction. VR easy mode enabled!

    0.6
    -Setting policy to optimize for hmd will automatically rotate position
    data
    -Fixed pinch detection stability, grabbing will now override pinching
    event
    -Changed event names to better reflect their purpose
    -Added gesture event, set gesture support at startup to receive this
    event
    -Added polling data for: interaction box, pointables, gestures, images
    (basic support)

    0.5.2
    -Added Event Driven architecture, Add LeapEventInterface to your
    blueprint class and call LeapController->SetDelegate(self). Your class
    will now receive leap events.
    -Added Arm support, please see official leap documentation for reference
    -Plugin now automatically converts leap space position data to UE space
    data including scale (mm->cm)
    -Added support for HMD Optimized tracking (also known as top down
    tracking). Set OptimizeForHMD(true) on your LeapController.

    0.5.1
    -Updated to Leap SDK 2.1.5+22693
    -Enabled getting finger data. Use Hand->Fingers

    0.5.0
    -compile fixes and updated to UE4.4

    Todo:
    -Launch build fix (launch standalone, PIE, and shipping/packaging work without issues)
    Last edited by getnamo; 02-14-2017 at 02:45 PM.

  2. #2
    0
    Hey man, came from the Leap forums! I like this, perfect for what I want to do. Am going to give it a try and see what I can do with it

    Your documentation and explanation is awesome by the way!

  3. #3
    0
    Infiltrator
    Join Date
    Apr 2014
    Posts
    19
    Hello. I have a question.

    This did not work for OptimizeForHMD set true.

    Will need to have the OVR?

  4. #4
    0
    Updated plugin to v0.6 with the following changes:

    -Setting policy to optimize for hmd will automatically rotate position
    data
    -Fixed pinch detection stability, grabbing will now override pinching
    event
    -Changed event names to better reflect their purpose
    -Added gesture event, set gesture support at startup to receive
    gesture events (refer to leap api)
    -Added polling data for: interaction box, pointables, gestures, images
    (basic support)


    Quote Originally Posted by simpple View Post
    Hello. I have a question.

    This did not work for OptimizeForHMD set true.

    Will need to have the OVR?
    This was working according to the leap api, it expected you to do rotations manually.

    Since this is the expected use case, I've added automatic rotation changes when you enable optimize for HMD, try again and let me know if that is what you wanted.

    Edit: 0.6 only adjusted for front facing rotation, 0.6.1 added HMD orientation and position adjustments as additional options for OptimizeForHMD.
    Last edited by getnamo; 10-12-2014 at 11:11 PM.

  5. #5
    0
    Quote Originally Posted by simpple View Post
    This did not work for OptimizeForHMD set true.
    First off thanks for the contribution! Hopefully between what you have going on here and what Leap has going on in their forums we'll eventually get an official LeapMotion plugin for UE4.

    Anyways, I am also developing for the Oculus Rift, and using the above blueprint event graph layout with OptmizeForHMD checked. The debug boxes only draw in any meaningful relation to my character when I look straight up at the ceiling where the Leap is in it's Non-HMD orientation. This could be tied to the fact that I'm using a first person template and not third person as you were, and may require some additional rotations/translations to place the debug spheres properly in relation to the headset.

    Note sure if it's relevant, but Leap (https://developer.leapmotion.com/vr) says, "starting with v2.1.3, you must set the policy flags POLICY_IMAGES and POLICY_OPTIMIZE_HMD to access the Image API and head-mounted tracking in your application".

    Hoping to have the time in the near future to start digging through their VRIntro source code.
    Last edited by spyderweb; 10-12-2014 at 07:17 PM.

  6. #6
    0
    Updated plugin to 0.6.1 which contains convenience options for VRMode.

    You will now see autoRotation and autoShift booleans in addition to useTopdown for OptimizeForHMD. Setting all three to enabled will make your reported leap positions automatically adjust for your head position and look direction.

    VR easy mode enabled!

    Note sure if it's relevant, but Leap (https://developer.leapmotion.com/vr) says, "starting with v2.1.3, you must set the policy flags POLICY_IMAGES and POLICY_OPTIMIZE_HMD to access the Image API and head-mounted tracking in your application".
    This is what OptimizeforHMD does. Image API is currently the only real feature not properly supported (may get to it at some point).

    Anyways, I am also developing for the Oculus Rift, and using the above blueprint event graph layout with OptmizeForHMD checked. The debug boxes only draw in any meaningful relation to my character when I look straight up at the ceiling where the Leap is in it's Non-HMD orientation. This could be tied to the fact that I'm using a first person template and not third person as you were, and may require some additional rotations/translations to place the debug spheres properly in relation to the headset.
    Try the auto-rotation and auto-shift options and let me know if that fixes things! If you're using the VR mode I suggest you shrink the debug spheres radius and scale to 1 as this will give that 1:1 feel.

  7. #7
    0
    Thank you so much for this! Finally an easy way to experiment with the leap and vr in ue4.

    The raw image support would be interesting for me only in the sense that it would enable a somewhat easy port of this which is essential for a tracking thing I'm trying. (opentrack use in ue4 is somewhat problematic for me)

  8. #8
    0
    Update to 0.6.2
    -Updated to UE4.5
    -Removed the need to SetInterfaceDelegate! Just adding the Component and
    adding the LeapEventInterface is enough.
    -Added Image policy flag support, actual image handling is still not
    working at this time. You can query about image dimension and other
    stats, just not the raw image.

    Updated documentation to reflect newer options and added some documentation regarding gestures. Full image support is missing at this time.

  9. #9
    0
    Hi,
    and thanks for sharing this plugin with us .
    However.. I'm currently having a problem with the plugin.. as soon as I add the interface to my controller (MyController) and launches the project I end up with a message saying "Fatal error!" and that the game crashed and the message that's displayed refers to a bunch of addresses and paths (as far as I can tell all the paths do exist).

    I have added the Bin and Plugin folders and all the files to my project, and the plugin is enabled.

    Im using the latest Leap SDK and UE4.4.3, I have also confirmed that my leap is working with other leap projects.
    Im not sure if it matters but Im using Windows 8.1 x64.. if that's helpful in some way .

    Any ideas?

  10. #10
    0
    Infiltrator
    Join Date
    Apr 2014
    Posts
    21
    Hi !! First I want to thank you very much for your plugin I was waiting a lot for that !!
    I was wondering if you could explain me if there is a way to allow the hands to follow the same rotation of the character Root (Capsule Component) ? ^^
    When i am rotating my character, the hand doesn't rotate :'[
    Thank you !
    Last edited by Xeno; 10-15-2014 at 06:20 PM.

  11. #11
    0
    Quote Originally Posted by Inx51 View Post
    Hi,
    and thanks for sharing this plugin with us .
    However.. I'm currently having a problem with the plugin.. as soon as I add the interface to my controller (MyController) and launches the project I end up with a message saying "Fatal error!" and that the game crashed and the message that's displayed refers to a bunch of addresses and paths (as far as I can tell all the paths do exist).

    I have added the Bin and Plugin folders and all the files to my project, and the plugin is enabled.

    Im using the latest Leap SDK and UE4.4.3, I have also confirmed that my leap is working with other leap projects.
    Im not sure if it matters but Im using Windows 8.1 x64.. if that's helpful in some way .

    Any ideas?
    Ok.. so I just tried the updated version of the plugin with UE4.5 and also changed my projects directory to the default path... but now I keep getting this error:
    Name:  Capture.PNG
Views: 19774
Size:  74.2 KB
    Any ideas?

    If its possible it would be really nice to have a sample project to test with to make sure that I have setup everything right
    Last edited by Inx51; 10-15-2014 at 07:01 PM.

  12. #12
    0
    Quote Originally Posted by Inx51 View Post
    Ok.. so I just tried the updated version of the plugin with UE4.5 and also changed my projects directory to the default path... but now I keep getting this error:
    ...
    Any ideas?

    If its possible it would be really nice to have a sample project to test with to make sure that I have setup everything right
    Just did a blank project (no code) test with the currently uploaded 0.6.2 plugin from this page (github download), it worked without problems.

    Please ensure you're installing this correctly



    Also check that you're using the Windows x64 platform (XP/Vista/7/8 x64), I have not tested this plugin for any other platform and will need to add relevant binaries before it will work. If both of those cases are correct and it is still not working, send me your logs (found in {ProjectRoot}/Saved/Logs)

    Quote Originally Posted by Xeno
    Hi !! First I want to thank you very much for your plugin I was waiting a lot for that !!
    I was wondering if you could explain me if there is a way to allow the hands to follow the same rotation of the character Root (Capsule Component) ? ^^
    When i am rotating my character, the hand doesn't rotate :'[
    Thank you !
    You have to rotate the positions you get from your leap by your control rotation like so


    Make sure the order you do this is correct as it will rotate the vector around its origin. In most cases this will be before you convert it to world space.

  13. #13
    0
    Infiltrator
    Join Date
    Apr 2014
    Posts
    21
    Thank you very much it's working perfectly !! Have a good day !!!

  14. #14
    0
    Hey, Is there anywhere where I can download an example project? Thanks, Lewis

  15. #15
    0
    Update to 0.6.3
    -Changed plugin to runtime to support shipping builds
    -See the github readme for shipping/packaging instructions.

    Quote Originally Posted by Inx51 View Post
    Ok.. so I just tried the updated version of the plugin with UE4.5 and also changed my projects directory to the default path... but now I keep getting this error:
    Any ideas?

    If its possible it would be really nice to have a sample project to test with to make sure that I have setup everything right
    This was happening because of launching v0.6.2 or lower version of the plugin which didn't have packaged/shipping support. Download 0.6.3 plugin or later and see the github readme for shipping/packaging instructions.

    Quote Originally Posted by lewisq View Post
    Hey, Is there anywhere where I can download an example project? Thanks, Lewis
    I don't plan on including an example project as it is two simple steps to include leap support. Follow the how to instructions and you will be up and running in a few minutes.
    Last edited by getnamo; 10-16-2014 at 05:27 PM.

  16. #16
    0
    Hi,

    I tried running the plugin in UE4.5 and I get message:
    The following modules are missing or build with a different engine version:
    UE4Editor-LeapMotion.dll
    Would you like to rebuild them now?

    When I click yes, it fails to compile.
    I copied Binaries and Plugin folders to the root of the project.
    I also tried the 4.4 version fo the plugin and get a diffrent message
    "Plugin 'LeapMotion' failed to load because module LeapMotion' could be loaded...'

    Any help will be much appreciated, I'm completly stuck!

  17. #17
    0
    Quote Originally Posted by BartBarlowski View Post
    Hi,

    I tried running the plugin in UE4.5 and I get message:
    The following modules are missing or build with a different engine version:
    UE4Editor-LeapMotion.dll
    Would you like to rebuild them now?

    When I click yes, it fails to compile.
    I copied Binaries and Plugin folders to the root of the project.
    I also tried the 4.4 version fo the plugin and get a diffrent message
    "Plugin 'LeapMotion' failed to load because module LeapMotion' could be loaded...'

    Any help will be much appreciated, I'm completly stuck!
    Ensure you're on Win64, make sure you download the latest plugin version (0.6.3) this is compiled for UE4.5. When you play hit play not launch. If you want to run it outside the editor use Play->Standalone Game. If you want to package/cook and test for shipping, read the shipping instructions.

  18. #18
    0
    Quote Originally Posted by getnamo View Post
    Ensure you're on Win64, make sure you download the latest plugin version (0.6.3) this is compiled for UE4.5. When you play hit play not launch. If you want to run it outside the editor use Play->Standalone Game. If you want to package/cook and test for shipping, read the shipping instructions.
    Thanks for the quick reply. I'm on Win64. Downloaded the latest plugin and UE4.5. Unfortunately I can't get to the editor (it just stops loading when the "The following modules are missing or build with a different engine version: UE4Editor-LeapMotion.dll Would you like to rebuild them now?" so I did not get to test it with play. Am I missing something?

  19. #19
    0
    Hey getnamo I'm putting the BP together but I noticed that I dont have a debug sphere at actor I can only bring up Draw Debug Sphere. Does that matter? Also I can only bring up stabilizedPalmPosition and stabilizedTipPosition, not palmPosition or tipPosition, I'm assuming you updated those nodes. Just thought I should mention.

    UPDATE: Nevermind debug sphere at actor is a custom function that I didn't scroll down far enough thank you though. I'm a newb so could you push me in the direction of adding actual bones to this system? Thanks getnamo! great job!!!
    Last edited by Joe1029; 10-16-2014 at 10:11 PM.

  20. #20
    0
    Update to 0.6.4
    -Fixed lack of categorization caused by updates in some of the blueprints classes
    -Cleaned up a lot of the blueprint functions, turning all the relevant
    ones into properties. This will reduce the number of execution pins.
    -Things which return a new object generally remain a function for
    potential performance reasons

    Quote Originally Posted by Joe1029 View Post
    ... Also I can only bring up stabilizedPalmPosition and stabilizedTipPosition, not palmPosition or tipPosition, I'm assuming you updated those nodes. Just thought I should mention...
    Yeah something I did in the earlier updates changed the blueprint categorization. I went through each blueprint class in the plugin and changed a lot of them into properties which should reduce the execution pin requirements.

    With the change I updated the quick setup and debug hand images which include each finger bone giving you something like this:



    Grab the updated plugin (0.6.4) and check at the begining of the thread for the updated API example.

    Thanks for the quick reply. I'm on Win64. Downloaded the latest plugin and UE4.5. Unfortunately I can't get to the editor (it just stops loading when the "The following modules are missing or build with a different engine version: UE4Editor-LeapMotion.dll Would you like to rebuild them now?" so I did not get to test it with play. Am I missing something?
    That error simply means your project is not in UE4.5, make sure to switch the engine version or create a new 4.5 project. I updated the plugin again, this time it was tested on a fresh no-code project and I confirmed it worked. Try re-downloading the latest plugin and start from the beginning.

  21. #21
    0
    Infiltrator
    Join Date
    Aug 2014
    Posts
    15
    Thanks for this great tool guys! I hope to contribute to the thread in a few weeks.
    Anyone have graphic card recommendations for best hmd/ue4/leap usage. I am still organizing myself for leap integration as my leap doesnt arrive for hmm well maybe tomorrow!
    Has anyone had any luck getting the raw images into ue4?
    Cant wait to share my project! Thanks again.

  22. #22
    0
    Hey Getnamo where do I get "TYPE" from? It's not in my library. I tried making an enum variable but couldnt connect it like you have it in the images.

  23. #23
    0
    Quote Originally Posted by rasamaya View Post
    Thanks for this great tool guys! I hope to contribute to the thread in a few weeks.
    Anyone have graphic card recommendations for best hmd/ue4/leap usage. I am still organizing myself for leap integration as my leap doesnt arrive for hmm well maybe tomorrow!
    Has anyone had any luck getting the raw images into ue4?
    Cant wait to share my project! Thanks again.
    Get a high-end gpu as VR is demanding, if you can afford it a 980gtx if not a 970gtx is currently the best sweet spot.

    Its also your lucky day, just updated to 0.7 which adds....

    Leap Image API
    Drag out nodes like this to obtain an image


    Which gives the following left image.



    For demonstration purposes the distortion map is also shown on the right. See the github repo and Leap Documentation for more details.

    The image is stable and real-time due to my texturing method containing no extra copies. The next step will be warping this around your head and trying to get something similar to VR Intro... but that may take some time, until then tinker away!

    Quote Originally Posted by Joe1029
    Hey Getnamo where do I get "TYPE" from? It's not in my library. I tried making an enum variable but couldnt connect it like you have it in the images.
    The plugin has been pretty active of late, some of those nodes have changed into properties but their names should be the same. Just make sure you start dragging out from the colored node (the object) not the execution node, from there just typing 'Leap' will narrow you execution choices to the ones relevant to the plugin.
    Last edited by getnamo; 10-18-2014 at 09:43 AM.

  24. #24
    0
    Lol dude I just updated the plug in today!

  25. #25
    0
    Has anyone been able to get the gesture sample hooked up? I can't find where "TYPE" is. I would really appreciate any help.

  26. #26
    0
    Updated the image for recognizing gestures which reflects the new property based setup. With the latest plugin version, just follow this graph:


  27. #27
    0
    Hi Getnamo.

    Great work on the plugin. Is there an example blueprint for using the skeletal mesh of the hero character instead of the debug spheres?

    Thanks

  28. #28
    0
    Quote Originally Posted by Synesthesia View Post
    Hi Getnamo.

    Great work on the plugin. Is there an example blueprint for using the skeletal mesh of the hero character instead of the debug spheres?

    Thanks
    See the hydra thread where we've talked about inverse kinematics to set this up for hands/arms. It is not a trivial topic, so ensure you understand how the unreal engine animation system works, essentially you just forward your leap motion positions to IK nodes (or FK rotations) in the animation graph, refer to IK setup documentation and animation documentation in general for details.

  29. #29
    0
    Quote Originally Posted by BartBarlowski View Post
    Hi,

    I tried running the plugin in UE4.5 and I get message:
    The following modules are missing or build with a different engine version:
    UE4Editor-LeapMotion.dll
    Would you like to rebuild them now?
    I have the same problem as BartBarlowski.

    I'm on Win 8.1 x64. My Editor is built from source. It says it is version 4.5.1. Anytime I add your folders to a project I can no longer open that project because of the above error.
    I'm now downloading the Engine provided by the launcher (4.5.0). I'll let you know if that makes a difference.

    However, the better solution would be if I could build the plugin myself. Can you please provide instructions on how to do that?

    Thank you!
    Last edited by bullale; 10-20-2014 at 11:51 AM.

  30. #30
    0
    Quote Originally Posted by bullale View Post
    I have the same problem as BartBarlowski.

    I'm on Win 8.1 x64. My Editor is built from source. It says it is version 4.5.1. Anytime I add your folders to a project I can no longer open that project because of the above error.
    I'm now downloading the Engine provided by the launcher (4.5.0). I'll let you know if that makes a difference.

    However, the better solution would be if I could build the plugin myself. Can you please provide instructions on how to do that?

    Thank you!
    To build the plugin from source, simply install the plugin as usual (drag and drop Plugins and Binaries into project root) and add code to your project. Delete the UE4Editor-LeapMotion.dll and then when you recompile it should include your plugin in the compilation as well.

    Alternatively you can add the plugin into the engine root in the same way (turning it into an engine plugin) and it will be enabled in all projects by default, but I haven't tested this myself.

  31. #31
    0
    Quote Originally Posted by getnamo View Post
    To build the plugin from source, simply install the plugin as usual (drag and drop Plugins and Binaries into project root) and add code to your project. Delete the UE4Editor-LeapMotion.dll and then when you recompile it should include your plugin in the compilation as well.
    Thank you. I was able to compile it with Unreal Engine 4.5.1

    I noticed that the event executions aren't visualized in the blueprint view. I don't know if that's expected or not.

    Do you know how I can find the real leap-character offset for use with the oculus rift? I can't seem to find a setting that feels natural. I still haven't figured out the character/actor/camera hierarchy.
    Do you have a DK2? If you do and it works well for you, would you be able to share your character BluePrint in your github repo? I'll keep working at it but any help is greatly appreciated.

  32. #32
    0
    Bullale I got the arms feeling natural in the oculus by playing with the vector values inside of the "draw sphere at actor" I think those values were around x=40 y=0= z= 40, the values in the examples were throwing my hands way down.

  33. #33
    0
    Infiltrator
    Join Date
    Aug 2014
    Posts
    15
    Nice work on the video input, I will be testing tonight, just got my leap today!

  34. #34
    0
    Has anyone been working on setting leap hands to mesh? I went to the hydra thread like getnamo said and studied the examples him and some other users posted along with the IK and animation docs. I thought I had an idea how to transport what I had seen in that thread over to the leap but I failed quite hilariously instead of following my hands like I hoped it would. Once either of my hands are detected myCharacter's hands shoot straight to its crotch pics below. Since I'm an all around noob for the moment I couldnt place the pictures like I wanted to but I will explain my process.

    First pic shows the event graph in my BP_LeapController. In it I made two events; Left Hand moved and Right hand Moved, to be honest I don't know why I set it up this way cuz all it is doing is drawing my spheres (I think) and then I Set Leap enabled. The Draw Debug Sphere function is exactly the same as @Getnamo example.

    Second picture is the HeroTPP_AnimBP's event graph, here I was hoping I would be able to get the location from BP_leapController such as left Wrist and right Wrist and SET the LeftHand and RightHand location. I think I'm not receiving any data at this point which would make sense why myCharacter is touching itself. (The wires outside the pic are from the regular animBP.)

    After that I set up both IK nodes for the left and right hand but since I don't really know if I'm receiving any data from the initial cast I am stumped if anything in here is messed up or not. Do I need to set up rotation?

    If anybody has a clue or is interested in figuring this out I would be happy to work together I just want to be able to see my hands in VR
    Attached Images Attached Images      

  35. #35
    0
    Name:  Error_UE4.PNG
Views: 19498
Size:  88.7 KB

    Hey guys, long time listener, first time caller.
    I was able to install the plug-in just fine.
    In the project, I was able to create via blueprint the Debug Hands example from the github page.
    However, once I exit the project and try to re-open it, the editor crashes, and I get the following error message.
    Any help would be super appreciated. Have a nice day!

  36. #36
    0
    Quote Originally Posted by Joe1029 View Post
    Has anyone been working on setting leap hands to mesh? I went to the hydra thread like getnamo said and studied the examples him and some other users posted along with the IK and animation docs. I thought I had an idea how to transport what I had seen in that thread over to the leap but I failed quite hilariously instead of following my hands like I hoped it would. Once either of my hands are detected myCharacter's hands shoot straight to its crotch pics below. Since I'm an all around noob for the moment I couldnt place the pictures like I wanted to but I will explain my process.

    First pic shows the event graph in my BP_LeapController. In it I made two events; Left Hand moved and Right hand Moved, to be honest I don't know why I set it up this way cuz all it is doing is drawing my spheres (I think) and then I Set Leap enabled. The Draw Debug Sphere function is exactly the same as @Getnamo example.

    Second picture is the HeroTPP_AnimBP's event graph, here I was hoping I would be able to get the location from BP_leapController such as left Wrist and right Wrist and SET the LeftHand and RightHand location. I think I'm not receiving any data at this point which would make sense why myCharacter is touching itself. (The wires outside the pic are from the regular animBP.)

    After that I set up both IK nodes for the left and right hand but since I don't really know if I'm receiving any data from the initial cast I am stumped if anything in here is messed up or not. Do I need to set up rotation?

    If anybody has a clue or is interested in figuring this out I would be happy to work together I just want to be able to see my hands in VR
    Your graph looks ok except for two things.

    One your Joint Target is 0,0,0 this is probably incorrect, joint target should be in the direction of your elbow, you can play around with it in your Persona Preview to find the right values.

    The second part is that you're forwarding the raw Leap Values. Consider for a moment what these are: the position of the hand in relation to the leap motion controller in the real world. What the animation graph is looking for is the position of the hands in relation to the character root component position. So you need to find the offset between Leap Real World->Character Root. If you're using the Leap default mode this offset is unknown. With the hydra we solved this using a calibration step which gave the offset of the hydra base to the shoulder midpoint which then got shifted down to the character root. This isn't really possible with the Leap in default mode since there isn't a calibration step which can determine where you body is in relation to the leap (maybe you could point toward your body and grab that point, but that sounds shaky at best), so your best bet is to have a fixed offset that the user can adjust.

    However: If you're using the Leap Motion mounted to your HMD, this is moot. Simply tick optimize for hmd, auto-shift, and auto-rotate and all your values will be in 1:1 in component space. This means you can forward the raw values straight to the skeleton. The reason this works is because it is mounted to your HMD which is tracked with a direct position, so the plugin figures out the offset and rotation adjustment to translate your values into component space automagically .


    Quote Originally Posted by bullale View Post
    Thank you. I was able to compile it with Unreal Engine 4.5.1

    I noticed that the event executions aren't visualized in the blueprint view. I don't know if that's expected or not.

    Do you know how I can find the real leap-character offset for use with the oculus rift? I can't seem to find a setting that feels natural. I still haven't figured out the character/actor/camera hierarchy.
    Do you have a DK2? If you do and it works well for you, would you be able to share your character BluePrint in your github repo? I'll keep working at it but any help is greatly appreciated.
    To my knowledge event visualization (blueprint debugging) doesn't happen for interfaces, ask epic to add that capability.

    Currently I do not have a character blueprint specific to the leap, I'm working on something more generalized which would allow a lot of hardware specific binding to be skipped, this may take some time though, until then explore the setup with the rest of the ue4 leapers

    NB: General Hierarchy for character animation is Character/Pawn: Leap Controller Event/Poll->OffsetValues, Animation Event Graph: Character:OffsetValues->Hand/Finger Values in Component Space, Animation Graph: Hand/Finger Values->IK Bone Effector location/FK bone rotation

    Quote Originally Posted by joemamaishere123 View Post
    ...

    Hey guys, long time listener, first time caller.
    I was able to install the plug-in just fine.
    In the project, I was able to create via blueprint the Debug Hands example from the github page.
    However, once I exit the project and try to re-open it, the editor crashes, and I get the following error message.
    Any help would be super appreciated. Have a nice day!
    From your crash output it doesn't seem to be related to the plugin. If you still suspect the plugin, you can compile the plugin (follow instructions earlier in the thread) to get a .pdb file which will give you more detailed output in your logs and in the crash output.

  37. #37
    0
    Well I am practically 100% certain the error is coming from the plugin because everything works fine until I add LeapMotion blueprints into the event graph. It can be anything as simple as a single LeapEvent, and the editor crashes with the above message.

    However, I may have found out why I am getting the above crazy error message.
    I tried to add the LeapEventInterface without adding any blueprint code, and I get the following error when re-launching the editor.

    Name:  Error_More_UE4.PNG
Views: 19172
Size:  22.6 KB

    The LeapEventInterface is then no longer included in the implemented interfaces.
    So I hypothesize that... the LeapEventInterface keeps getting removed and then when UE4 tries to "read" the LeapMotion blueprints, it crashes because the interface isn't there.
    Do you have any idea why I am getting this new error message? And why the LeapEventInterface is being removed?

  38. #38
    0
    Quote Originally Posted by joemamaishere123 View Post
    Well I am practically 100% certain the error is coming from the plugin because everything works fine until I add LeapMotion blueprints into the event graph. It can be anything as simple as a single LeapEvent, and the editor crashes with the above message.

    However, I may have found out why I am getting the above crazy error message.
    I tried to add the LeapEventInterface without adding any blueprint code, and I get the following error when re-launching the editor.

    Name:  Error_More_UE4.PNG
Views: 19172
Size:  22.6 KB

    The LeapEventInterface is then no longer included in the implemented interfaces.
    So I hypothesize that... the LeapEventInterface keeps getting removed and then when UE4 tries to "read" the LeapMotion blueprints, it crashes because the interface isn't there.
    Do you have any idea why I am getting this new error message? And why the LeapEventInterface is being removed?
    I think you might have some left over functions hanging around in the event graph. When you remove LeapEventInterface from a blueprint that depends on it, it gives you an option to remove them or convert to custom nodes. Then when you add the interface again it will clash since the names are the same.

    I recommend starting from a fresh blueprint (copy a character) and add/remove things until you stop getting these errors. I've tried but I cannot reproduce it myself, try checking your logs found in {Project Root}/saved/logs. By default the plugin warns you if you have a Leap Controller component, but no interface and having just an interface is harmless as it would simply not cause your events to be called.

  39. #39
    0
    Hi,

    first of all thanx for your work.
    I'm trying to use your plugin and so far everything's fine but...
    How can I intercept a SwipeGesture and use its variables in my blueprint ? I looked quickly in your C++ and found nothing.
    Is there a way ?

  40. #40
    0
    Oh no, there are no leftover functions in the event graph or anything. It's a fresh new project. It's a new FPS C++ template project. I go into the MyCharacter blueprint and ONLY try to add the LeapEventInterface in implemented interfaces. It saves and compiles fine. But then when I exit the project, and re-open it, I get the above error.

    Later, if I try to add even a single LeapEvent or anything to do with the LeapMotion in the EventGraph, it, too, saves and compiles fine. But upon exiting and trying to re-open I get the original error where the editor doesn't even open.

    I realize if you can't reproduce the error, there is not much you can do. But hopefully you can help because this is maddening.

    I should note the LeapEventInterface isn't being removed by ME. I receive the above error message and the LeapInterface is automatically removed.

Page 1 of 14 12311 ... LastLast

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •