Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • started a topic [Plugin] Leap Motion - Event Driven

    [Plugin] Leap Motion - Event Driven

    The official Leap Motion plugin for the Unreal Engine 4.

    You can use convenience blueprints to just select and play or use a custom approach via blueprints or C++. See relevant sections in https://github.com/leapmotion/LeapUnreal for details.

    NB: Only supports Windows 64 & 32

    Download

    Latest: https://github.com/leapmotion/LeapUnreal/releases

    Older
    Engine (4.11-4.18) - It's already in the engine, just enable and restart!
    0.9.9 for UE4.10 / Leap SDK 2.3.1 - Mac Binaries synced to 4.10

    Resources
    Github - Main Plugin Source and Documentation
    Leap Motion C API documentation - For C++ programmers who want finer control
    Leap Community UE4 Plugin Thread - Old leap motion forum thread for v2 plugin

    How to Use



    See https://github.com/leapmotion/LeapUnreal


    Changelist
    Latest changes available at: https://github.com/leapmotion/LeapUnreal/releases

    3.0.0
    -
    Switching off leap service should no longer crash on exit
    - Ensure project plugin overrides old LeapC.dll
    - Updates and fixes for epic engine code
    - Switched to Async module for threading
    - Update internal API string to 4.0
    - Track motion controllers in bodystate, add custom device tag filters for selective animation consumption
    - Add support for Leap Image API from v4 SDK, events available from leap component. Requires setting appropriate policy
    - Add fix for #4 using scope locks

    2.17.1
    - First LeapUnreal release
    - Updated Leap SDK to v4.0 with improved tracking

    2.15
    - Compile fixes for 4.19. Uses new IXRTrackingSystem
    - Now works as both engine and project plugin. Will auto-detect environment and adjust dependencies.
    - Engine plugin requires IWYU only. All includes changed to this preference.
    - BS Confidence value fixes for 4.0 sdk, will work for 3.2 as well
    - misc fixes and doc changes

    2.12
    - Added BodyStateBone Component. Similar to Motion Controller components, but it will track any body state bone from a drop down list.
    - Added BodyStateEstimator Component, which allows merging and updating skeleton data in a separate component which gets fed into the final merged skeleton. Useful for estimating body positions from incomplete data.
    - Fixed position offsets not being overwritten when using default tracking modes

    2.11.2
    - Fixed compile warning for packaged builds
    - added log forwarding from leap service
    - enabled hot-restarting the leap service
    - added more verbose logging for problem states

    2.11.0
    - Fixed and added auto-rigging for much wider mesh setups. See https://github.com/getnamo/leap-ue4-...custom-rigging for documentation.
    - Exposed global position and rotation offsets used to bring alignment 1:1
    - Fixed #28
    - Fixed #31
    - Added #30
    - Fixed #29

    2.10.0
    - Compile fixes for 4.18

    2.9.3
    - Compile fixes for 4.17
    - Fixed bodystate for packaged game
    - Added finger extension state to BSFinger

    2.9.0
    - Default rigging set to bodystate low poly
    - significant changes to body state anim graph collapsing all blueprint code into a single node using a mapped data structure
    - body state skeleton auto-set on animation start
    - first pass at auto-mapping arbitrary bones to bodystate bones.
    - editor friendly method to adjust auto-mapped results in defaults, allowing for easy fine tuning
    - hand rigging now deforms to match your tracked data
    - time warp updates to sync with newly deformed hands due to now using bone positions

    2.8.0
    - Renamed module from LeapMotionC to LeapMotion, issue #19
    - Misc body state offset fixes to sync behavior with low poly direct rigging

    2.7.0
    - Compile fixes for 4.16
    - Compile fixes for #16
    - Updated documentation

    2.6.7
    - Merged as b46ad8d
    - Oculus Timewarp fixes, the platform should have good and stable timewarp default settings
    - Three mode tracking optional settings allowing the choice of whether you want low latency, smooth, or balanced tracking
    - All Leap settings and options as a FLeapOptions struct for more finer grained control available globally
    - Added FLeapStats for polling information about frame extrapolation and device details, also available globally

    2.6.1
    - Working timewarp implementation
    - Leap frame interpolation and extrapolation added
    - Default settings to extrapolate to current gamethread time + 5.5ms TW
    - More optimization settings have been exposed
    - Profiling cycle counters added
    - Fixes and tweaks

    2.4.2
    -First Pre-Release with proper packaging support.

    2.0 - 2.4
    -Internal development builds

    2.0.0
    -Integrated directly in 4.11 engine
    -Added Image Hands
    -Added Component Hands
    -Thorough code commenting
    -Automatic packaged support
    -Added convenience LeapFloatingHandsCharacter

    0.9.9
    -Updated Mac binary for 4.10

    0.9.8
    -Updated binaries to UE4.10

    0.9.7
    -Fixed packaging for 4.9.2
    -Added EnableBackgroundTracking toggle on LeapController allowing you to
    track when the window isn't focused
    -Updated Echo hands to hide when not being tracked, idle
    animations for Echo hands have been removed. Use
    LeapEchoHandsRiggedCharacter for a rigged mesh hand without attached
    body, use the collision variant for physics enabled version.

    0.9.6
    -Binaries synced with UE 4.9
    -Leap SDK synced with 2.3.1
    -Added Echo convenience hands (rigged floating hands)

    0.9.5
    -Updated binaries to UE4.8

    0.9.4
    -Updated Leap SDK to 2.2.6

    0.9.3
    -Added LeapCollisionCharacter which adds physics collision enabled by
    default. This can be enabled for other setups by using the PhysicsActor
    collision preset.
    -Updated PhysicsAsset used for collision
    -Removed old content
    -Fixed SphereRadius property link found on LeapHand

    0.9.2
    -Mac Binaries compiled for UE 4.7
    -Leap SDK, libs and dlls updated for SDK 2.2.4

    0.9.1
    -Cleaned up inheritence structure, LeapAnimBodyActor renamed to
    LeapAnimBodyConnector. Added Passthrough sub-class
    -LeapPassthroughCharacter now contains the subclass. Rigged characters
    contain no passthrough code
    -Fixed elbow rotation bug

    0.9
    -Added properly HMD warped image passthrough support through both
    materials and a convenience character (LeapPassthrougCharacter)
    -128bit texture support resolved for Distortion textures. Use Distortion
    for proper float data, use DistortioUE to visualize the texture for
    debug purposes.
    -Optional content moved into plugin, delete if you don't use it and wish
    to save space
    -Rigging update to be more loosely coupled. Character anim blueprint
    only depends on AnimBody obtained through a simple and safe
    AnimBodyInterface
    -Rigged character finger rotation bug fixed
    -Updated leap bind to 2.2.3, much improved tracking gained.
    -Added R->Reset position to rigged characters, change key bind to
    reflect own bindings

    0.8.3
    -Now supports Mac OSX
    -Fixed Finger->Hand crash on exit
    -Update to Leap SDK 2.2.1

    0.8.2
    -Now supports UE 4.6

    0.8.1
    -Fixed Memory leaks which would cause crash around 60second mark (GC
    interval) for list-based objects using extended and append
    -Added appendTools and appendFingers to pointableList
    -LeapFrame got its API updated to include all of the missing functions,
    should now reflect the leap api nearly completely (except 64bit values
    in blueprint that are not supported)
    -Added log entry with plugin version at game start for easier debugging

    0.8.0
    -Basic Rigged leap character added. Works like the LeapDebugCharacter,
    change your player controller to VRPlayercontroller and pawn to
    LeapRiggedCharacter
    -Updated bind to Leap SDK 2.2.0
    -Ensured consistency by prepending leap to all classes, this will likely
    break compatibility with old blueprints.
    -Fixed dynamic object allocation architecture
    -Added basic AnimBody, Hand, Finger, and Bone for convenient animation
    state and basis conversion.

    0.7.11
    -You can now check which finger you have by dragging out the Type
    property from Finger.

    0.7.10
    -Added a convenience Leap Debug Hand actor, allowing for easy display of
    debug hands
    -Added a convenience Leap Debug Character which is set up to show debug
    hands in both HMD and Default mode. Simply toggle your hmd (alt+enter)
    to test HMD mode.

    0.7.9
    -added Input Mapping support for pinch grab and palm orientation for
    each hand. See documentation for details
    -added PalmOrientation convenience function to Leap Hand. Origin is
    defined as palm flat facing down, fingers pointing away.

    0.7.8
    -Update to Leap SDK 2.1.6. Includes changes from leap
    (https://developer.leapmotion.com/doc...#version-2-1-6)

    0.7.7
    -If you tick auto-shift on hmd, this will automatically add the
    eye->leapmount offset (8cm default value). Your vr hands should now be
    completely 1:1
    -Added Hand Type enum to Hand and relevant gestures. You can now easily
    switch the hand type instead of checking isLeft/Right to reduce bp
    clutter.
    -Added Leap Plugin Direction (Cardinal Direction) to gestures which
    emit directions (Screen Tap, Key Tap, and Swipe). This is a convenience
    property which will allow you to easily switch the property to get basic
    swipe left/right/up/down/towards/away detection. If you need a more
    precise direction, use the direction vector.

    0.7.6
    -Added Raw Image events which give much cleaner support and memory
    efficient toggling of streaming images from the device.
    -Multiple components are now properly supported. Reason for error was
    mismanaged GC of on-demand UObject subclasses. Plugin will still emit
    multi-component warning for performance reasons (redundancy/duplication).
    -All logging switched to LeapPluginLog sub-category

    0.7.5
    -At the moment multiple components cause a crash after 40-60sec of
    detecting a hand/finger, until this error is resolved, the plugin will
    output a warning to your logs if you add more than one total leap
    component

    0.7.4
    -Normalized vectors such as palm normal were not properly adjusted for
    HMD view direction, this is has now been fixed.

    0.7.3
    -palm normal scaling fixed.

    0.7.2
    -Circle gesture radius now in UE scale
    -Swipe Gesture speed now in UE scale

    0.7.1
    -Hidden Leap header and private implementation from UClasses
    -Fixed image stability, will now correctly resize when requested.
    -R8 texture no longer causes memory leaks
    -Added Gesture sub-classes and corresponding events (e.g. Swipe Gesture
    Detected). Will allow for easy determination of direction and other
    properties of gestures. You can also cast general gesture detection to
    the correct sub-class gesture.
    -Added own category of logging, pay attention to log output to help you
    with debugging errors.

    0.7
    -Added full support for Image API! see post or github readme for an example use case.
    -Fixed a lot of memory leaks, should have no more memory leaks and purr like a kitten.

    0.6.4
    -Fixed lack of categorization caused by previous updates in some of the blueprints classes
    -Cleaned up a lot of the blueprint functions, turning all the relevant
    ones into properties. This will reduce the number of execution pins.
    -Things which return a new object generally remain a function for
    potential performance reasons

    0.6.3
    -Changed plugin to runtime to support shipping builds
    -See the github readme for shipping/packaging instructions.

    0.6.2
    -Updated to UE4.5
    -Removed the need to SetInterfaceDelegate! Just adding the Component and
    adding the LeapEventInterface is enough.
    -Added Image policy flag support, actual image handling is still not
    working at this time. You can query about image dimension and other
    stats, just not the raw image.

    0.6.1
    -OptimizeForHMD now contains 2 more booleans, one for auto rotation
    adjustment and one for auto position adjustment. Setting all three to
    enabled will make your reported leap positions automatically adjust for
    your head position and look direction. VR easy mode enabled!

    0.6
    -Setting policy to optimize for hmd will automatically rotate position
    data
    -Fixed pinch detection stability, grabbing will now override pinching
    event
    -Changed event names to better reflect their purpose
    -Added gesture event, set gesture support at startup to receive this
    event
    -Added polling data for: interaction box, pointables, gestures, images
    (basic support)

    0.5.2
    -Added Event Driven architecture, Add LeapEventInterface to your
    blueprint class and call LeapController->SetDelegate(self). Your class
    will now receive leap events.
    -Added Arm support, please see official leap documentation for reference
    -Plugin now automatically converts leap space position data to UE space
    data including scale (mm->cm)
    -Added support for HMD Optimized tracking (also known as top down
    tracking). Set OptimizeForHMD(true) on your LeapController.

    0.5.1
    -Updated to Leap SDK 2.1.5+22693
    -Enabled getting finger data. Use Hand->Fingers

    0.5.0
    -compile fixes and updated to UE4.4
    Last edited by getnamo; 08-23-2018, 05:21 PM.

  • replied
    Originally posted by SaraCarraro View Post
    Hi.
    Thank you for the plugin!

    I started this leap motion "journey" downloading the lastes vesion
    available:
    4.22 plugin. I'm having some problems finding the plugin in Unreal, I can see only "magic leap" plugin in the plugin setting, that is not leap motion plugin. I put the Plugin folder in my project folder and i did the lip motion tryal in leap motion control panel, everyting seems ok with it.

    I can't understand how to activate the plugin in Unreal.
    Thank you in advance.

    The engine plugin is disabled by default, but you should be able to enable it by going Edit->Plugins and then selecting the category Input Devices. In there you'll find the Leap Motion Plugin tick enable and restart your project to see it loaded up.



    Optionally you can install it as a project plugin by getting the latest release from github e.g. https://github.com/leapmotion/LeapUn...ses/tag/v3.3.1 and extracting LeapMotion-v3.3.1-UE4.22.7z Plugins folder into your project root.
    Last edited by getnamo; 07-28-2019, 03:25 PM.

    Leave a comment:


  • replied
    Originally posted by Juangea View Post
    Hey getnamo any news on supporting the Oculus Quest device?
    That would depend on Leap Motion supporting the device. Best bet to track LM news would be https://www.leapmotion.com/news/

    Leave a comment:


  • replied
    Originally posted by Syed View Post
    Hi,

    I am trying to compile myself as my project is 4.19 (cannot upgrade for now) - managed to sort out the build.cs. Except for one c++ compilation error which is:-

    Code:
    ProducerLambdaFuture.Reset();
    The error was:-
    Code:
    class "TFuture<void>" has no member "Reset"
    Apparently UE4 4.19 doesn't have the member function Reset (maybe less advanced lol) so I just commented it. Is it okay?
    Should be ok, but with the caveat that it if you run your app without a leap motion service running it may be left hanging in the taskbar (it shouldn't but there is a risk for that particular case). That particular section is used to forcibly cleanup in the case of the leap service doesn't respond to polling (which is a v4 leap service bug we're working around).

    Leave a comment:


  • replied
    Hi,

    I am trying to compile myself as my project is 4.19 (cannot upgrade for now) - managed to sort out the build.cs. Except for one c++ compilation error which is:-

    Code:
    ProducerLambdaFuture.Reset();
    The error was:-
    Code:
    class "TFuture<void>" has no member "Reset"
    Apparently UE4 4.19 doesn't have the member function Reset (maybe less advanced lol) so I just commented it. Is it okay?

    Leave a comment:


  • replied
    Hi all - this might have been obvious to some, but certainly wasn't for me. When downloading any .7z module or plugin from GitHub, it is essential that you unzip the download with 7-zip: https://www.7-zip.org/ in order for everything to extract cleanly. Using WinZip, ALZip or other alternatives won't work properly.

    Leave a comment:


  • replied
    Hey getnamo any news on supporting the Oculus Quest device?

    Leave a comment:


  • replied
    Hi.
    Thank you for the plugin!

    I started this leap motion "journey" downloading the lastes vesion
    available:
    4.22 plugin. I'm having some problems finding the plugin in Unreal, I can see only "magic leap" plugin in the plugin setting, that is not leap motion plugin. I put the Plugin folder in my project folder and i did the lip motion tryal in leap motion control panel, everyting seems ok with it.

    I can't understand how to activate the plugin in Unreal.
    Thank you in advance.

    Leave a comment:


  • replied
    Map: DesktopHelloWorld
    Left and Right hand movement are ok.

    Adding LeapMannequinMerged_Anim

    Right Hand move the arm Crossing middle body coordinate. 0 or 180 degrees.

    I'm going to read your recomandencaiones to get it. Thank you very much for your time and links. I really appreciate it
    Attached Files

    Leave a comment:


  • replied
    Originally posted by Carlosan View Post
    Sorry if I waste your time and thanks for your patience.
    I followed all your steps but I still get this when moving my right hand.

    L to R
    LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim
    I think I see the problem. Currently it's setup to have a standard tracking where left = left etc, but you want a mirror mode where you right hand is the character's left hand facing you. In your particular example, if you rotated your characters 180 degrees they would line up with your tracked hands.

    To do a mirror mode, you'd need to reflect the tracking data about a Z plane (math: http://mathworld.wolfram.com/Reflection.html but you can likely do vector subtraction from a fixed point for simpler math). There's nothing built in that supports it in e.g. bodystate, instead I'd recommend starting with tracking data from a leap component, reflecting that data and feeding it into a mesh. See the direct rigging examples on how to use the leap component based setup https://github.com/leapmotion/LeapUn...igging/LowPoly

    Leave a comment:


  • replied
    Sorry if I waste your time and thanks for your patience.
    I followed all your steps but I still get this when moving my right hand.

    L to R
    LeapMannequinFull_Anim - LeapMannequinMerged_Anim - LeapMannequinAutoHandsOnly_Anim

    Attached Files

    Leave a comment:


  • replied
    Originally posted by Carlosan View Post
    Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

    But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

    Thx for your time.
    repo uses git lfs, if you do not use git lfs, download a release (.7z) at https://github.com/leapmotion/LeapUn...ses/tag/v0.3.7. If you clone without lfs you may end up with an empty directory.

    Leave a comment:


  • replied
    Desktop Mode - Example Actor is working fine using 4.22.1. Thx.

    But https://github.com/leapmotion/LeapUn...odules/Rigging blueprint are lost using latest version. Cant find it anymore inside content folder (using latest commit uploaded at github 1 day ago)

    Thx for your time.

    Leave a comment:


  • replied
    Originally posted by Carlosan View Post
    Yes, using latest release 0.3.7
    If i move Right hand, i see on the display the left hand moving. Same with arms
    Pic attached. Thx
    Desktop mode? You should be using https://github.com/leapmotion/leapun...-example-actor which sets the optimization for desktop mode (and changes the expected orientation of the leap motion device)

    Confirm you get good tracking by trying out the DesktopHelloWorld.umap in https://github.com/leapmotion/LeapUnrealModules

    Leave a comment:


  • replied
    Yes, using latest release 0.3.7
    If i move Right hand, i see on the display the left hand moving. Same with arms
    Pic attached. Thx
    Attached Files

    Leave a comment:

Working...
X