Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Thanks I got it working! I initially wasnt able to get the rotation from the palm but thanks to the API I found out I had to Frame > Hand > Rightmost > Palm Normal and Palm Direction > Break Rot > Make Rot.

    Now I need to work on the finger IK any suggestions?

    Xeno I've posted all the blueprints to get the hands matched to the position if you look at the past couple of posts I have three pics that show everything that I have done. First I'd say get the debug spheres working cuz that really helped me to understand and visualize how leap data is being transferred. Once you have that check out the past posts, if you fall into some trouble ask away I'll try to help.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    if anyone wants to know how I got this far I will supply you pics, just let me know.
    I am curious to see your nodes
    I'm little confused :/
    Thank you x)

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Getnamo thanks for the update!!! Once I have my hands working I'm gonna create some spell casting shenanigans!!!!

    What is the proper way to grab rotation data from the leap and port over to mesh hands?
    It terms of how you're sampling the rotation it appears correct. I'm not sure what the fabrik rotation does (haven't used it myself yet), if you're having trouble using it, you could instead simply rotate the characters hand bone using FK (transform bone) after you do the hand IK.

    Do note that you will likely have to rotate your leap rotations such that they are in the bone space. For the hydra, I adjusted to the local space using something like



    There should be more details on this in the hydra thread. You can change axis directions or swap them around using break rot, but all rotation transforms should be combined using combine rotators, which uses quaternions in its implementation, this ensures you do not have a gimbal lock by converting axis.

    Leave a comment:


  • replied
    Getnamo thanks for the update!!! Once I have my hands working I'm gonna create some spell casting shenanigans!!!!

    What is the proper way to grab rotation data from the leap and port over to mesh hands?

    Leave a comment:


  • replied
    Update to 0.7.2
    -Circle gesture radius now in UE scale
    -Swipe Gesture speed now in UE scale
    (0.7.1)
    -Hidden Leap header and private implementation from UClasses
    -Fixed image stability, will now correctly resize when requested.
    -R8 texture no longer causes memory leaks
    -Added Gesture sub-classes and corresponding events (e.g. Swipe Gesture
    Detected). Will allow for easy determination of direction and other
    properties of gestures. You can also cast general gesture detection to
    the correct sub-class gesture.
    -Added own category of logging, pay attention to log output to help you
    with debugging errors.

    The plugin is now properly encapsulated and includes various memory leaks and bug fixes. Also added all the missing sub-gestures, if you've been looking for gesture sub-classes such as SwipeGesture or CircleGesture, this is the update you've been waiting for.

    Example of displaying circle gesture's you're making, either of the two cases can be used, they are equivalent after the generic gesture is cast.

    Leave a comment:


  • replied
    Ok so now I'm working on adding rotation to it. This is what I have so far but I'm not getting any rotation to my hand.

    To get the rotation from my palm position I do a rotation from Xvector then I set a Right Hand Rotation variable.
    After that I get the Right hand rotation from my leapController BP and set the right hand rot inside my AnimBP event graph
    Then in my AnimGraph I grab my right hand position and my right hand rotation plug that into a make transform and then all that goes into a FABRIK.

    Since I know it doesn't rotate my hand I know it's not working. Does any one have any suggestions? Also if anyone wants to know how I got this far I will supply you pics, just let me know.
    Attached Files

    Leave a comment:


  • replied
    Wow getnamo thank you I did these changes and immediately was able to bend my elbows in persona. to have it running in game I had to do some math to calculate for the mesh offset. I'll be making a tutorial of some kind once I actually get both arms working right.

    So now I need to add the leap rotation data. How would I go about doing that? I tried getting the palm rotation but could figure out how to get that. I'll keep pluggin away at it. Thanks!

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Hey Getnamo so I've been playing around with the IK and also FABRIK thanks to n008 in the hydra forum but I've noticed that my hands do not move in the Y axis which is my forward and back or toward the skeleton and away. Both n008 and I were wondering if maybe the data is scaled too far out in front of the player and therefore would stretch out the arms not allowing the bones to bend. Do you think this might be a possibility. I'm just doing guess work at this point.
    I went back to my IK implementation for the hydra and I realized that I was misrepresenting a few facts. I'll break these down into topics

    1. Actor origin vs component space origin. Whenever you get Actor->Get location, it will give you the location of your actor origin (0,0,0), which in the default character is close to the pelvis, but not in any way related to the pelvis bone. The component space on the other hand will give you the origin of your mesh, which for the default character is at his feet. I thought my implementation was using component space, but I realized I had automatically added the world location and used world space for my effector locations.

    So if you want to use component space for effector location you need to find the mesh location transform relative to the actor origin (mesh->location) and add that offset to your hand positions, or if you use world space for effector location, you need to add the actor world location to your hand positions.

    2. Joint Targets, its important to specify the space you use these in, I recommend using component space or something that follows the body form such as spine_03. The proper way should be using the upper arm bone space, but this is very difficult to calibrate correctly in my experience.

    Also if you're doing animations which involve your hands I recommend blending out your IK contribution for the duration or you may get 'elbow jitter', using a stable space such as component will minimize this type of animation blending jitter.

    Below is a working example.


    In this particular case:
    1. Left Hand Location and Right Hand Location values are actor origin offset hydra values converted into world space (Hydra in Actor Origin + Actor Location).
    2. Joint Targets are in spine_03 bone space, they are fixed, set as default values and never changed. You can see the spine_03 axis in the persona preview
    3. Hand Locations are just values I changed to show poses where the elbows bend naturally. Right hand fist is closed using finger IK.

    Please note that I do not consider this example to be anywhere near perfect, but something that has worked for my testing purposes. Here's an example of the previous setup which had component space joint targets using a Razer Hydra and an Oculus Rift DK2: Gfycat Album

    @Ryunigia
    Still working on fixing this class of bug, its an architecture bug which only affects the Leap Plugin. I haven't had the time to properly address it.

    Regarding the temporary fix, if you want to use C++ for gameplay, do that. Subclassing a blueprint child and adding the leap component there should work. You also have the option of adding the leap component to a simple blueprint actor and forwarding the values to your pawn/character.
    Last edited by getnamo; 10-25-2014, 01:50 PM.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    Ok found a temporary workaround. Create a fps template, duplicate the MyCharacter in case you mess up in the future (as far as I know the bp is not recoverable after you add the component or interface), now for the fix:

    1.Sub-class the Mycharacter in blueprint which by default will make a MyCharacter_Child.
    2.Add your leap interface and leap component to this child class.

    This will put a layer of indirection to your C++ class and the leap blueprint and it ... works.

    I believe this stems from the origins of the plugin which retained a direct link to the Leap header and put all the classes into private implementation. These are still visible in blueprint, but I think C++ cannot include them properly. I will need to rethink how the architecture is built and re-implement it in such a way that the classes can be publicly exposed and included in C++ source. Until then just sub-class your C++ class and use that child to implement leap functions.
    ok i just wanted to say, i tried this on 4.51 and i didnt work for me :\
    so i switched to the blueprint fps project and everything worked fine ever since. (same bug as joemamaishere123) to bad though :s
    not sure if i did anything wrong i just created a new blueprint derived from MyCharacter and used that child to implement all the leap functions (interface/controller).
    either way thanks for the plugin and the response ^^'

    Leave a comment:


  • replied
    Hey Getnamo so I've been playing around with the IK and also FABRIK thanks to n008 in the hydra forum but I've noticed that my hands do not move in the Y axis which is my forward and back or toward the skeleton and away. Both n008 and I were wondering if maybe the data is scaled too far out in front of the player and therefore would stretch out the arms not allowing the bones to bend. Do you think this might be a possibility. I'm just doing guess work at this point.

    Leave a comment:


  • replied
    Getnamo, thanks for all the advice by the way. I get that the preview was just to test out the values and that the default is where those values actually live but what I was saying is that whether I am in preview or default the values that I change don't affect the skeleton in any way. For example I set all the joint values to 10000 just to see some results and nothing happened there was no bending even in unnatural ways. When I get to the office I'll test it out with your numbers but I'm afraid the problem above will still persist. Based on my IK set up from the earlier pictures do you think I could have set something wrong?

    EDIT:
    Hey so I just used your values in the default area but still the elbows do not bend. What is your Effector Location Space? Mine is in component space so is my Joint target location space. I'm playing with different variations of these as well but I still don't get any bend. Could you also show me which bones are part of your IK system? Here is my set up the left matches the right.
    Attached Files
    Last edited by Joe1029; 10-24-2014, 01:06 PM.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Getnamo, I figured out why my characters hands were staying next its hips was that the leap data was not being transferred 1:1 even though the "Use Topdown" was ticked. So I had to break the vectors flip the X and Y and times the X and Z by 100 and Y by -100. So now my arms are moving about but I still do not get any bend in my elbows I've played with the preview and default values with the Leap enabled and still nothing happens to the Joint target location. No matter what value I put into joint target location my joint target is not being affected. Do you have any idea what could be happening?
    Remember that the preview values are just that, for preview. Once you find a good left/right joint location you should copy those into their default values. Generally the joint targets shouldn't need to change.

    In your preview with certain joint positions, your elbows will bend, and at a good range they will bend naturally. Since we're using the same model, you can probably use the same values as I have in the image I posted earlier. Do remember that the character 'forward vector' is by default in the Y direction in the animation blueprint.

    Leave a comment:


  • replied
    Getnamo, I figured out why my characters hands were staying next its hips was that the leap data was not being transferred 1:1 even though the "Use Topdown" was ticked. So I had to break the vectors flip the X and Y and times the X and Z by 100 and Y by -100. So now my arms are moving about but I still do not get any bend in my elbows I've played with the preview and default values with the Leap enabled and still nothing happens to the Joint target location. No matter what value I put into joint target location my joint target is not being affected. Do you have any idea what could be happening?
    Attached Files
    Last edited by Joe1029; 10-23-2014, 08:34 PM.

    Leave a comment:


  • replied
    Originally posted by Inx51 View Post
    me.... again....
    Im trying to create a packaged build, but Im abit unsure of how to do this.
    I have read your documentation but I dont quite follow...

    Projects require code, if you are using blueprint only add an empty class and compile your project module.
    What sort of empty class?.. a C++ empty class?.. any specific class-name or filename thats required?.. in which dir should I place this empty class?.. how do I compile it?..
    So for instance in my case I have used only Blueprints and used the FPS template without downloading any engine source code.. is ths engines source-code required for the compile?
    If there is any other documentation regarding this that I could read then please let me know

    Br,
    Inx
    You simply do File->Add Code to Project and it can be anything so usually I just pick None->Create Class and then it will ask you to open visual studio where you just hit compile (Build solution or F7).

    If you haven't added code before follow the unreal engine programing Quick Start guide. Essentially it boils down to downloading the free Visual Studio express and changing a few small configs.

    You absolutely do not need a copy of the UE engine source, I almost exclusively use the stable prebuilt binaries for the editor that unreal releases since my code doesn't modify the engine in any way.

    Originally posted by Inx51
    Hi again getnamo!

    Im trying to determine which direction a user "swipes".. for instance.. from left->right or from right->left.. but I cant quite figure out how to do this the best way.. I guess that since I have the last 60 frames I could compare the hands position over the last for instance 20 frames and determine, do you have any other better options?

    Thanks in advance!
    You can identify the direction by simply comparing the last frame hand/finger/pointable position to the current one.

    But! I realized I have not included the gesture sub-classes in the plugin! Will update the plugin to include those and then it should be as easy as Gesture->Cast to Swipe Gesture-> Direction.

    Leave a comment:


  • replied
    me.... again....
    Im trying to create a packaged build, but Im abit unsure of how to do this.
    I have read your documentation but I dont quite follow...

    Projects require code, if you are using blueprint only add an empty class and compile your project module.
    What sort of empty class?.. a C++ empty class?.. any specific class-name or filename thats required?.. in which dir should I place this empty class?.. how do I compile it?..
    So for instance in my case I have used only Blueprints and used the FPS template without downloading any engine source code.. is ths engines source-code required for the compile?
    If there is any other documentation regarding this that I could read then please let me know

    Br,
    Inx

    Leave a comment:

Working...
X