Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by getnamo View Post
    Ok found a temporary workaround. Create a fps template, duplicate the MyCharacter in case you mess up in the future (as far as I know the bp is not recoverable after you add the component or interface), now for the fix:

    1.Sub-class the Mycharacter in blueprint which by default will make a MyCharacter_Child.
    2.Add your leap interface and leap component to this child class.

    This will put a layer of indirection to your C++ class and the leap blueprint and it ... works.

    I believe this stems from the origins of the plugin which retained a direct link to the Leap header and put all the classes into private implementation. These are still visible in blueprint, but I think C++ cannot include them properly. I will need to rethink how the architecture is built and re-implement it in such a way that the classes can be publicly exposed and included in C++ source. Until then just sub-class your C++ class and use that child to implement leap functions.
    ok i just wanted to say, i tried this on 4.51 and i didnt work for me :\
    so i switched to the blueprint fps project and everything worked fine ever since. (same bug as joemamaishere123) to bad though :s
    not sure if i did anything wrong i just created a new blueprint derived from MyCharacter and used that child to implement all the leap functions (interface/controller).
    either way thanks for the plugin and the response ^^'

    Leave a comment:


  • replied
    Hey Getnamo so I've been playing around with the IK and also FABRIK thanks to n008 in the hydra forum but I've noticed that my hands do not move in the Y axis which is my forward and back or toward the skeleton and away. Both n008 and I were wondering if maybe the data is scaled too far out in front of the player and therefore would stretch out the arms not allowing the bones to bend. Do you think this might be a possibility. I'm just doing guess work at this point.

    Leave a comment:


  • replied
    Getnamo, thanks for all the advice by the way. I get that the preview was just to test out the values and that the default is where those values actually live but what I was saying is that whether I am in preview or default the values that I change don't affect the skeleton in any way. For example I set all the joint values to 10000 just to see some results and nothing happened there was no bending even in unnatural ways. When I get to the office I'll test it out with your numbers but I'm afraid the problem above will still persist. Based on my IK set up from the earlier pictures do you think I could have set something wrong?

    EDIT:
    Hey so I just used your values in the default area but still the elbows do not bend. What is your Effector Location Space? Mine is in component space so is my Joint target location space. I'm playing with different variations of these as well but I still don't get any bend. Could you also show me which bones are part of your IK system? Here is my set up the left matches the right.
    Attached Files
    Last edited by Joe1029; 10-24-2014, 01:06 PM.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Getnamo, I figured out why my characters hands were staying next its hips was that the leap data was not being transferred 1:1 even though the "Use Topdown" was ticked. So I had to break the vectors flip the X and Y and times the X and Z by 100 and Y by -100. So now my arms are moving about but I still do not get any bend in my elbows I've played with the preview and default values with the Leap enabled and still nothing happens to the Joint target location. No matter what value I put into joint target location my joint target is not being affected. Do you have any idea what could be happening?
    Remember that the preview values are just that, for preview. Once you find a good left/right joint location you should copy those into their default values. Generally the joint targets shouldn't need to change.

    In your preview with certain joint positions, your elbows will bend, and at a good range they will bend naturally. Since we're using the same model, you can probably use the same values as I have in the image I posted earlier. Do remember that the character 'forward vector' is by default in the Y direction in the animation blueprint.

    Leave a comment:


  • replied
    Getnamo, I figured out why my characters hands were staying next its hips was that the leap data was not being transferred 1:1 even though the "Use Topdown" was ticked. So I had to break the vectors flip the X and Y and times the X and Z by 100 and Y by -100. So now my arms are moving about but I still do not get any bend in my elbows I've played with the preview and default values with the Leap enabled and still nothing happens to the Joint target location. No matter what value I put into joint target location my joint target is not being affected. Do you have any idea what could be happening?
    Attached Files
    Last edited by Joe1029; 10-23-2014, 08:34 PM.

    Leave a comment:


  • replied
    Originally posted by Inx51 View Post
    me.... again....
    Im trying to create a packaged build, but Im abit unsure of how to do this.
    I have read your documentation but I dont quite follow...

    Projects require code, if you are using blueprint only add an empty class and compile your project module.
    What sort of empty class?.. a C++ empty class?.. any specific class-name or filename thats required?.. in which dir should I place this empty class?.. how do I compile it?..
    So for instance in my case I have used only Blueprints and used the FPS template without downloading any engine source code.. is ths engines source-code required for the compile?
    If there is any other documentation regarding this that I could read then please let me know

    Br,
    Inx
    You simply do File->Add Code to Project and it can be anything so usually I just pick None->Create Class and then it will ask you to open visual studio where you just hit compile (Build solution or F7).

    If you haven't added code before follow the unreal engine programing Quick Start guide. Essentially it boils down to downloading the free Visual Studio express and changing a few small configs.

    You absolutely do not need a copy of the UE engine source, I almost exclusively use the stable prebuilt binaries for the editor that unreal releases since my code doesn't modify the engine in any way.

    Originally posted by Inx51
    Hi again getnamo!

    Im trying to determine which direction a user "swipes".. for instance.. from left->right or from right->left.. but I cant quite figure out how to do this the best way.. I guess that since I have the last 60 frames I could compare the hands position over the last for instance 20 frames and determine, do you have any other better options?

    Thanks in advance!
    You can identify the direction by simply comparing the last frame hand/finger/pointable position to the current one.

    But! I realized I have not included the gesture sub-classes in the plugin! Will update the plugin to include those and then it should be as easy as Gesture->Cast to Swipe Gesture-> Direction.

    Leave a comment:


  • replied
    me.... again....
    Im trying to create a packaged build, but Im abit unsure of how to do this.
    I have read your documentation but I dont quite follow...

    Projects require code, if you are using blueprint only add an empty class and compile your project module.
    What sort of empty class?.. a C++ empty class?.. any specific class-name or filename thats required?.. in which dir should I place this empty class?.. how do I compile it?..
    So for instance in my case I have used only Blueprints and used the FPS template without downloading any engine source code.. is ths engines source-code required for the compile?
    If there is any other documentation regarding this that I could read then please let me know

    Br,
    Inx

    Leave a comment:


  • replied
    Hi again getnamo!

    Im trying to determine which direction a user "swipes".. for instance.. from left->right or from right->left.. but I cant quite figure out how to do this the best way.. I guess that since I have the last 60 frames I could compare the hands position over the last for instance 20 frames and determine, do you have any other better options?

    Thanks in advance!

    Leave a comment:


  • replied
    Originally posted by BartBarlowski View Post
    Finally got it work! I had to manually copy Leap.dll from the Win64 folder to my Windows/System32 folder. You should add this to the install instructions.
    You shouldn't need to do that. This meant you probably used a non-standard version of the UE editor and you needed to compile the plugin, or you didn't add your leap.dll in the proper place (binaries in project root) or you were launching your app (which isn't really supported with plugins) instead of playing in editor (or play standalone).

    I suppose this is one way to get that form of use working though.

    Originally posted by zalo View Post
    The reason the distortion maps appear in the green and blue channels has to do with the pixel format you're using ("PF_B8G8R8A8") in the CreateTransient function (orders it BGRA).

    You want to use "PF_R8G8B8A8" to get them into proper RGBA... though you should really be using "PF_G32R32F" (32-Bit Floats in each of the Red and Green channels) for lossless distortion data.

    EDIT: Hmm, looks like my bit-fu isn't up to the task...
    I tried this when I was doing the image conversion for 0.7 and decided it wasn't worth the extra effort for now, but if you can make the 32-bit floats work, please make a pull request and I'll merge it. It also doesn't matter that the distortion map is split into specific channels since they are simply specific distortion information which you will pull out of the channels when you use the pixel and fragment shader to correct the image for whatever projection you want to use (see Leap Camera Image API).
    Last edited by getnamo; 10-26-2014, 01:42 PM.

    Leave a comment:


  • replied
    The reason the distortion maps appear in the green and blue channels has to do with the pixel format you're using ("PF_B8G8R8A8") in the CreateTransient function (orders it BGRA).

    You want to use "PF_R8G8B8A8" to get them into proper RGBA... though you should really be using "PF_G32R32F" (32-Bit Floats in each of the Red and Green channels) for lossless distortion data.

    EDIT: Hmm, looks like my bit-fu isn't up to the task...
    Last edited by zalo; 10-22-2014, 04:54 PM.

    Leave a comment:


  • replied
    Finally got it work! I had to manually copy Leap.dll from the Win64 folder to my Windows/System32 folder. You should add this to the install instructions.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Ok Getnamo update on my failures. In an attempt to get the joint target set up I created sockets at each elbow and placed the sockets 60 units behind each elbow (respectively 60 and -60) this is what I understood as behind the elbow. Then I put those value in the Y axis' of my joint variables plugged that into the joint target location and still nothing. I notice that when I launch the game the characters hands are still around its waist but the hands are receiving input from the leap so they move around near the waist but do not lift up . My hands are not moving in the z axis.
    Couple of things (and this is why skeletal animation isn't really a trivial topic). Make sure you have rotated your inputs to have Y as forward (whereas world uses X), make sure again you are not just forwarding the raw values, but that you have a leap to character root offset added to the inputs before they are forwarded (unless you're using topdown with an hmd). The joint targets should probably be in component space and should not rotate with your arm movements (it may work otherwise, but I haven't personally tested this)

    In your preview it appears that your hands are close to your character pelvis which indicates the alpha is working, since 0,0,0 is your component root which in the default character is at the pelvis. Now all you need to do to preview is change the Left Hand and Right Hand Position and see how the elbow bends as you move the positions around. Adjust the joint targets such that the elbow bends in a natural way.

    Also ideally you should set Alpha to 0 whenever you hands are not visible, since you will have wrong data otherwise. That way you can let your animations handle your hand positions when they aren't being tracked by the leap. Quite a lot of this is covered in the hydra thread. Debug your vectors to visualize what they are doing and try to make them match what the animation system is expecting (by seeing what preview values are needed for the hand motions), this will indicate what you're doing wrong.

    I am working on a generalized structure, which will have a convenience skeletal bind which will work for all vr input types, but it may be a few weeks before that gets posted, until then refer to the past few posts, the hydra thread and the unreal IK documentation for the how-to knowledge.

    Leave a comment:


  • replied
    Ok Getnamo update on my failures. In an attempt to get the joint target set up I created sockets at each elbow and placed the sockets 60 units behind each elbow (respectively 60 and -60) this is what I understood as behind the elbow. Then I put those value in the Y axis' of my joint variables plugged that into the joint target location and still nothing. I notice that when I launch the game the characters hands are still around its waist but the hands are receiving input from the leap so they move around near the waist but do not lift up . My hands are not moving in the z axis.
    Last edited by Joe1029; 10-21-2014, 10:32 PM.

    Leave a comment:


  • replied
    So I set up leftJoint and rightJoint variables and I was tweaking the numbers but nothing happens and I made sure my alpha was on which is being driven by Bool LeapEnabled which is checked. Is it possible that something else is broken?
    Attached Files

    Leave a comment:


  • replied
    hey anyone can make a small video for show the actual and real latency?

    Leave a comment:

Working...
X