Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by aretoon View Post
    Hey Getnamo,

    First of all, thank you so much for all that you're doing for this community! You are responding trying to help out anyone with their issues, and it is inspiring

    I got my studio interested in Leap Motion recently and we're lloking to dig into the complex world of UE4 and want to use your plug in. So I'm sure we're going to have some questions, just to let ya know! haha.

    Anyway keep up the great work, thanks a million and looking forward to trying your plugin!
    Thanks for your kind words! Vr is a very new and dynamic field and I think by sharing and helping each other out we can all benefit from making better standards, gaining deeper knowledge, and most importantly getting that ease of use.


    Update to 0.7.9
    -added Input Mapping support for pinch grab and palm orientation for
    each hand. See documentation for details
    -added PalmOrientation convenience function to Leap Hand. Origin is
    defined as palm flat facing down, fingers pointing away.

    This may not be relevant for full vr, but adds a convenience for those who want to use the leap in basic ways by using the input mapping system for simple, joystick-like maneuvers.

    E.g. adding one input mapping key and 2 axis to the Rolling Template, allows you to roll the ball by the pitch and roll of your hand and jump by closing your fist. Simple fun!




    Edit:
    Update to 0.7.10
    -Added a convenience Leap Debug Hand actor, allowing for easy display of
    debug hands
    -Added a convenience Leap Debug Character which is set up to show debug
    hands in both HMD and Default mode. Simply toggle your hmd (alt+enter)
    to test HMD mode.

    Now you can add the optional content folder to your project root and use the LeapDebugHand and/or the LeapDebugCharacter to see how a simple display is achieved.


    To try out the debug character, simply change your default pawn


    The character has two LeapDebugHand objects and swaps dynamically between them to display the debug hands in both default mode (leap on table facing up) and HMD mode




    This should give you all some reference to easily test your setup. Will be working on a rigged model soonish(tm).
    Last edited by getnamo; 11-13-2014, 04:09 PM.

    Leave a comment:


  • replied
    Hey Getnamo,

    First of all, thank you so much for all that you're doing for this community! You are responding trying to help out anyone with their issues, and it is inspiring

    I got my studio interested in Leap Motion recently and we're lloking to dig into the complex world of UE4 and want to use your plug in. So I'm sure we're going to have some questions, just to let ya know! haha.

    Anyway keep up the great work, thanks a million and looking forward to trying your plugin!

    Leave a comment:


  • replied
    Originally posted by ipas View Post
    I also am puzzeling with the handrotation now. The flipping of the hand is because the ZX rotation flips from 180 to -180. I needed to solve this to convert the leaphand rotation to the mesh rotation of the hand. I use a timeline to be able to manipulate these conversion. First I add 180 to kind off convert the -180->180 degrees to a clean 0->360 'timeline' Then I plot the graph to get nice rotation degrees I can use for my mesh without flipping. Now I need to continue with yaw and pitch.. same thing i guess. ...
    Interesting, sounds like the orientation has its cross over point near the neutral position, but I cannot be certain. You can solve this in a couple of ways:
    1) Change the crossover point by adding offset orientation (which seems to be what you're doing). 2) Count revolutions and compensate by adding or subtracting 360 degrees.

    This is a bit weird since the hydra orientation did not have this issue, will check into this myself when I'll be making the body input plugin, but that may be a while.


    Originally posted by Posthuman View Post
    I'm trying to get the Leap plugin for Unreal Engine to work, but I'm getting an error when building:
    [2014.11.08-22.27.17:039][ 0]LogModuleManager:Warning: ModuleManager: Module 'LeapMotion' not found - its StaticallyLinkedModuleInitializers function is null.
    [ATTACH=CONFIG]16601[/ATTACH]

    I tried downloading v.2.1.5.22693 Beta instead of the newest v.2.1.6.23110 Beta, but the download on the leapmotion.com site looks broken...
    ,,,

    The Leap Motion Plugin is enabled in UE 4.5.1 project:
    ...

    Before extracting "leap-ue4-master.zip" to the root of the project I've also tried to unblock the zip (helps in some rare cases):
    ...

    Running Windows 8 Pro...

    Any ideas how to get my Leap working for VR ?

    Really can't wait to start messing around with this...
    I'm guessing you're trying to 'Launch' from the editor, this is not supported because the packing process does not automatically copy the required dlls, a limitation of plugins with dependencies which epic hasn't given guidance on how to remedy.

    What is supported is Play in Editor (PIE), Play Standalone, and Shipping and Packaged builds, for those see this post

    Edit:
    Update to 0.7.8
    -Update to Leap SDK 2.1.6. Includes changes from leap
    (https://developer.leapmotion.com/doc...#version-2-1-6)

    Key line from their change list: VR improvements. Note that with a Leap SDK update you will also need to update your Binaries folder (for regular updates only Plugins folder needs to be replaced).
    Last edited by getnamo; 11-10-2014, 02:06 AM.

    Leave a comment:


  • replied
    I'm trying to get the Leap plugin for Unreal Engine to work, but I'm getting an error when building:
    [2014.11.08-22.27.17:039][ 0]LogModuleManager:Warning: ModuleManager: Module 'LeapMotion' not found - its StaticallyLinkedModuleInitializers function is null.
    Click image for larger version

Name:	YNhkaoM.png
Views:	1
Size:	22.3 KB
ID:	1060784

    I tried downloading v.2.1.5.22693 Beta instead of the newest v.2.1.6.23110 Beta, but the download on the leapmotion.com site looks broken...
    Click image for larger version

Name:	84knVHb.png
Views:	1
Size:	31.4 KB
ID:	1060785

    The Leap Motion Plugin is enabled in UE 4.5.1 project:
    Click image for larger version

Name:	bq84uHL.png
Views:	1
Size:	33.4 KB
ID:	1060786

    Before extracting "leap-ue4-master.zip" to the root of the project I've also tried to unblock the zip (helps in some rare cases):
    Click image for larger version

Name:	0GLXtVp.png
Views:	1
Size:	35.5 KB
ID:	1060787

    Running Windows 8 Pro...

    Any ideas how to get my Leap working for VR ?

    Really can't wait to start messing around with this...

    Leave a comment:


  • replied
    I also am puzzeling with the handrotation now. The flipping of the hand is because the ZX rotation flips from 180 to -180. I needed to solve this to convert the leaphand rotation to the mesh rotation of the hand. I use a timeline to be able to manipulate these conversion. First I add 180 to kind off convert the -180->180 degrees to a clean 0->360 'timeline' Then I plot the graph to get nice rotation degrees I can use for my mesh without flipping. Now I need to continue with yaw and pitch.. same thing i guess. Click image for larger version

Name:	HandRotationTimeline.jpg
Views:	1
Size:	82.6 KB
ID:	1060761

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Doesn't the BP we have sets the direction of the hand to be pointing outwards towards the fingers. That's what the API says anyways.

    How would I go about combining rotators? I know I get one rotate from the break/make rot but where would I get the other? Would I just need to make another rotate node with the values you put above? (0, 0, -90)
    The api outputs both the forward vector and the palm normal vector. Make rot from ZX, which we use, gives you an orientation from the palm normal. If you want the vector to point forward use different axis or combine rotators to make it point outward. Debug the orientation and experiment until you get the axis you want (maybe XZ? haven't tested this).

    Given that actor space has x as forward, whereas anim blueprints have y as forward the second rotator should probably be (0,0,90) (Roll,Pitch,Yaw) which should turn the X forward into Y forward (turn right).
    Last edited by getnamo; 11-05-2014, 11:29 PM.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    Two possible cases for this problem:

    One: leap automatically tries to detect your hand direction, but sometimes it may guess wrong, try to move your hand in and out of view with slightly bent fingers and it should determine the direction correctly.

    Two: You may be breaking rot change values around which may cause a gimbal style lock. Remember that yaw and roll range from -180 to 180 and pitch is only from -90 to 90. General rule is you can invert and swap axis with break rot, but change orientation only with quaternion/matrix math. To change orientation in blueprint you should use combine rotators e.g. rotate axis left would be combine rotators(rotator1,rotator2) where rotator2 is 0,0,-90.
    Doesn't the BP we have sets the direction of the hand to be pointing outwards towards the fingers. That's what the API says anyways.

    How would I go about combining rotators? I know I get one rotate from the break/make rot but where would I get the other? Would I just need to make another rotate node with the values you put above? (0, 0, -90)

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Hey Getnamo the axis flipping is still happening. I updated the plug in to 7.6 and I still have the same problem. Any suggestions?
    Two possible cases for this problem:

    One: leap automatically tries to detect your hand direction, but sometimes it may guess wrong, try to move your hand in and out of view with slightly bent fingers and it should determine the direction correctly.

    Two: You may be breaking rot change values around which may cause a gimbal style lock. Remember that yaw and roll range from -180 to 180 and pitch is only from -90 to 90. General rule is you can invert and swap axis with break rot, but change orientation only with quaternion/matrix math. To change orientation in blueprint you should use combine rotators e.g. rotate axis left would be combine rotators(rotator1,rotator2) where rotator2 is 0,0,-90.

    Originally posted by Inx51 View Post
    Hi!
    Im not sure if this is entierly the same problem as described above.. but I sort of guess it is..
    I have setup the sphere-debugger blueprints and everything works fine, but for some reason my left arm is controlling the right arm in UE4 and my right arm is controlling the left arm..

    any ideas of how to make the right arm control the right arm and the left the left arm?


    Br,
    Inx
    Leap motion automatically detects and categorizes hands, you can always check which hand you have by grabbing the Is Hand Left/Right property on the hand object. Regarding your problem, You may have your view rotated by 180 degrees with respect to the actor orientation.

    Originally posted by Inx51 View Post
    Hi everyone,
    Im trying to figure out how to get the debug hand to not be oriented around the center of the player.. or well.. as it is now.. the hands are not "freely" moving around in the world, but they are attached to the rotation of the character.. is there anyway to make em feel more like they are just floating in the world?.. Im not entirely sure of how to explain what Im after, so sorry if this might seem like a strange question :/... but hopefully someone gets it..
    or actually.. basically what Im after is the same implementation that Unity has of the leap motion, the hands are not attached to the characters rotation but its feels more like they are actually a part of the world.
    if you were using



    to debug your hands, then simply delete the rotate vector node and pass the value straight through. This was merely added as a convenience and shouldn't be used when using hmd mode as it automatically shifts and rotates to compensate for hmd view. Additionally the position offset shouldn't be used in hmd mode either or in the case where you know the leap-actor offset in the table/default mode (update it with your real-world offset).

    Edit:
    Minor update to 0.7.7
    -If you tick auto-shift on hmd, this will automatically add the
    eye->leapmount offset (8cm default value). Your vr hands should now be
    completely 1:1
    -Added Hand Type enum to Hand and relevant gestures. You can now easily
    switch the hand type instead of checking isLeft/Right to reduce bp
    clutter.
    -Added Leap Plugin Direction (Cardinal Direction) to gestures which
    emit directions (Screen Tap, Key Tap, and Swipe). This is a convenience
    property which will allow you to easily switch the property to get basic
    swipe left/right/up/down/towards/away detection. If you need a more
    precise direction, use the direction vector.
    Last edited by getnamo; 11-04-2014, 11:58 PM.

    Leave a comment:


  • replied
    Hi everyone,
    Im trying to figure out how to get the debug hand to not be oriented around the center of the player.. or well.. as it is now.. the hands are not "freely" moving around in the world, but they are attached to the rotation of the character.. is there anyway to make em feel more like they are just floating in the world?.. Im not entirely sure of how to explain what Im after, so sorry if this might seem like a strange question :/... but hopefully someone gets it..
    or actually.. basically what Im after is the same implementation that Unity has of the leap motion, the hands are not attached to the characters rotation but its feels more like they are actually a part of the world.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Hey Getnamo the axis flipping is still happening. I updated the plug in to 7.6 and I still have the same problem. Any suggestions?
    Hi!
    Im not sure if this is entierly the same problem as described above.. but I sort of guess it is..
    I have setup the sphere-debugger blueprints and everything works fine, but for some reason my left arm is controlling the right arm in UE4 and my right arm is controlling the left arm..

    any ideas of how to make the right arm control the right arm and the left the left arm?


    Br,
    Inx
    Last edited by Inx51; 11-04-2014, 11:11 AM. Reason: Solved itself

    Leave a comment:


  • replied
    Hey Getnamo the axis flipping is still happening. I updated the plug in to 7.6 and I still have the same problem. Any suggestions?
    Last edited by Joe1029; 11-03-2014, 08:55 PM.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

    Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


    EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
    Update to 0.7.4
    -Normalized vectors such as palm normal were not properly adjusted for
    HMD view direction, this is has now been fixed.

    Updated the plugin because I realized that the normalized vectors which only used the convertLeapToUE function (such as the palm normal/direction) did not adjust for hmd use (topdown view needs to have it's axis rotated so that the perspective remains correct) this has been fixed.

    So with 0.7.4 you just need to use the above example I showed earlier and it will be correct even when you tick optimize for hmd.

    Edit:
    Update to 0.7.6
    -Added Raw Image events which give much cleaner support and memory
    efficient toggling of streaming images from the device.
    -Multiple components are now properly supported. Reason for error was
    mismanaged GC of on-demand UObject subclasses. Plugin will still emit
    multi-component warning for performance (once per play) reasons (redundancy/duplication).
    -All logging switched to LeapPluginLog sub-category

    The multiple component crash is now completely fixed. The reason for the delayed crashing was because garbage collection appeared to use 60 seconds as its interval, so the plugin was crashing because of mismanagement of my custom UObject references (created once and updated per need instead of recreating each frame for performance considerations), this has now been properly cleaned up.

    Decided the leap image polling api was a bit messy so added a convenient event for getting raw leap images

    it used to look like this


    you can still use that method but you can now do something much cleaner using events, such as


    both of which give the following billboard results (note the finger and palm debug visualization is a result of a separate graph)


    To receive the events and to enable image support in general call enable image suppport on your Leap Controller component


    if you use the images intermittently (e.g. fading real world in and out over the virtual one), you should toggle the image settings when they not in use to reclaim memory, performance, and help minimize usb bandwidth use.

    Example of a toggle
    Last edited by getnamo; 11-03-2014, 12:58 AM.

    Leave a comment:


  • replied
    Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

    Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


    EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
    Attached Files
    Last edited by Joe1029; 11-01-2014, 08:13 PM.

    Leave a comment:


  • replied
    Originally posted by machinimated View Post
    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
    Good catch. Currently multiple component use is undefined. I'm hoping to turn these into some sort of multi-cast in the future, which would allow you to have multiple components without issue, but for now just use one component


    Originally posted by mattplu View Post
    I'm hoping someone can offer some advice. I'm trying to create actual spheres instead of the debug spheres so they can collide with objects. I haven't found a way yet to create a static sphere and pick out the specific joint to attach each sphere to (most likely the best solution if possible).

    Instead I modified the Draw Debug Sphere function to spawn a blueprint actor with sphere instead and then set the lifespan to something extremely short. I also tried using an event tick to delete all actors of that blueprint class. It works for a while but eventually UE4 leaves a ghost of the deleted actor in the scene outliner and it crashes.

    Any one experience the same or found a way to create a simple set of spheres and move them rather than respawn (basically how to get the location of each and every joing LEAP provides).

    Thanks in advance!
    I missed this earlier in the thread, there are various ways to make this work. Remember one important rule in programming: creating and destroying is expensive, re-use where possible. One way would be to initially create the necessary spheres and then move/hide them based on use. I did something similar earlier, but beware the graphs are a bit complicated and not optimal (and tbh should be in C++), but as an example of drawing a 'fast sphere':



    and then hiding the ones no longer in use by a tick function like this



    BPList is an array of FastSphereBP (your sphere actor which can have collision). This is, again, not an ideal solution, but something I did to increase FPS for an oculus/hmd demo and shows one type of approach in getting this to work.

    Leave a comment:


  • replied
    Originally posted by machinimated View Post
    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
    Great!!! Is working now! Thanks machinimated!!!

    Leave a comment:

Working...
X