Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Using your plugin with 4.8 and everything seems work fine. Thanks for it.

    But i have 2 questions.

    1. Leap motion tracking is a little bit laggy compared to official one. Is there any way to improve it?

    2. Moving working through player's forward vector. I want to make it rift's rotation like on first person template from official demo pack, but couldn't find how.

    Thank you again for the plugin. Amazing work.

    Leave a comment:


  • replied
    Originally posted by dougb View Post
    Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

    I have tested both the Jenga level that getnano built as well as developed my own level.
    Okay, that's a bit different. Are you still experiencing the black screen in VR?

    For the VR hands, can you verify that the leap motion device goes into HMD mode? Can you try the leap motion apps in HMD mode without any issues?

    Can you also verify that your Leap Motion device is not mounted upside down on your rift?

    Leave a comment:


  • replied
    Originally posted by Slayemin View Post
    So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

    The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

    I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.
    Thanks Slayemin, but that's just it...the custom char skeleton that comes with the plug, works fine until I enter Rift mode. In Rift mode, my left and right thumbs face downward and if I move my physical arms up, the arms in the level move down and vice-versa. It's as if the skeletal is upside down. When not in Rift mode, everything works fine.

    I have tested both the Jenga level that getnano built as well as developed my own level.

    Leave a comment:


  • replied
    Originally posted by dougb View Post
    Fantastic plug-in! Thanks so much getnamo!

    One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

    Any idea?

    Thanks!
    So... I am using this plugin with oculus rift and got it working with hands from a custom character skeleton and mesh. Whatever you read about this not working with VR is wrong.

    The first thing you want to do is make sure that the sample out-of-the-box skeleton included within the plugin works correctly with the VR headset. It should, I've tested and verified it myself.

    I also ran into an issue where the headset goes black. After some investigating, I found that it has nothing to do with the plugin, but with the VR template's player controller I was using. I found that if you move your character into a volume area, the VR player controller template has a blueprint section which will apply a post processing effect which makes the screen go black. It's supposed to display a warning, but it doesn't. If you press "F2" to use the non-lit view, you should see the world just fine. I just disabled the offending section within the VR player controller blueprint and it ran fine (4th sequence pin within VR Controllers Tick function). You might be experiencing the same issue.

    Leave a comment:


  • replied
    Fantastic plug-in! Thanks so much getnamo!

    One problem though, when I create a new project or test the Jenga one, everything works fine until I enter Rift mode. Once I do that, my hands become inverted. Now, I did read that you can't use VR mode and only Window, but when I do that, and then enter Rift mode (fullscreen), it goes black. I've tried it on three computers and all the same result. I'm using 4.8, 0.6 runtime for Oculus and the latest Leap drivers. I've used direct and extended mode.

    Any idea?

    Thanks!

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).
    Ok, thanks! Unfortunately at the moment on Mac I cannot even build the engine from source due to this bug (mac is not top priority for Epic either, even the official Leap plugin is not yet available for mac in 4.8)

    Leave a comment:


  • replied
    Okay, I've figured out my issues.

    For anyone else who attempts to use a custom mesh and skeleton, make sure that the orientation of each bone exactly matches the orientation in the reference rig. Also, pay very close attention to bone weights on the fingers or you'll get messed up fingers.

    The finger bones on my right hand were inverted along the X axis and the arm bones were correct, so changing the basis wasn't going to help fix the problem.

    Leave a comment:


  • replied
    Originally posted by devel.bmad View Post
    Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

    ...
    I'll try to get mac compiled over the weekend. That said give compiling it a shot, you have the full source and it would be more sustainable for you as you won't have to wait for me to keep up (its not my primary platform).

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    That's because the precompile for 4.8 wasn't available yet. This has now been fixed.

    Update to 0.9.4
    -Updated Leap SDK to 2.2.6

    Update to 0.9.5
    -Updated binaries to UE4.8

    get the newest master zip from github to upgrade.
    Hi Getnamo! Still getting this error message, was the Mac version updated to 4.8 too?

    Click image for larger version

Name:	Screen Shot 2015-06-17 at 22.59.30.png
Views:	1
Size:	94.3 KB
ID:	1080019

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    I'm glad you've asked the question, I was hoping someone would dig deep enough to ask about the implementation. The change basis function rotates all the AnimBody bones (that are parented, so if you rotate the body it will rotate all the bones) to a new basis. Pre-rotation applies input rotation before you're in the new space so it will affect all of your input rotations. In contrast post-rotation only rotates it by a constant amount and will not affect the basis of the input.

    The reason for doing all of this is that in component space (e.g. character blueprints) X is forward Y is right, Z is up; but in animation Y is forward, X is left, Z is up. So in order for everything to appear correct we need to change the rotation space to the new Y = Forward orientation. That is all that function does.
    Okay, I get the FABRIK and Two bone IK stuff. I'm still really stuck on the basis change stuff though.

    I've got the left hand working perfectly, but I'm using different left prebase values and post base values:
    Left Prebase: 90,0,0
    Left Postbase: 0,0,0

    I don't know why, but that works. Your values are different:
    Left Prebase: -90,0,0
    Left Postbase: 0,0,-90

    The right hand is different though, and I'm stuck at the relationships between rotations:

    I'm using the exact same values you're using within the LeapBodyConnector and I get wrong results:
    Right Prebase: 90,0,180
    Right Postbase: 0,0,90


    Through some testing, I was able to find:
    Right Prebase: -90,0,-180
    Right Postbase: 0,0,0

    This puts my elbow and palm in the correct position, but the right hand fingers are rotated 180 degrees (roll) in the wrong direction, and the thumb goes into the palm.

    What's causing the difference in our pre and post basis values?

    Leave a comment:


  • replied
    Originally posted by Rasa View Post
    Hi Getnamo,

    While Inherting your plugin with recent rlease UE-4.8 raises error, [ATTACH=CONFIG]43826[/ATTACH]

    How to rectify this, Exepecting ASP Getnamo. . .
    That's because the precompile for 4.8 wasn't available yet. This has now been fixed.

    Update to 0.9.4
    -Updated Leap SDK to 2.2.6

    Update to 0.9.5
    -Updated binaries to UE4.8

    get the newest master zip from github to upgrade.

    Leave a comment:


  • replied
    How to enable this plugin with UE-4.8

    Hi Getnamo,

    While Inherting your plugin with recent rlease UE-4.8 raises error, Click image for larger version

Name:	error.JPG
Views:	1
Size:	21.1 KB
ID:	1079937

    How to rectify this, Exepecting ASP Getnamo. . .

    Leave a comment:


  • replied
    Originally posted by spyro View Post
    How to uninstall this plugin? I deleted all references, set the plugin to false in the config but still got a crash when restarting the project...
    Inside your project root: Delete the LeapMotion plugin folder in Plugins and remove the leap specific binaries in your project (leap.dll, msvcr120.dll and msvco120.dll in Binaries/Win64 and Win32). Then it will be easy to clear the references from there.

    Leave a comment:


  • replied
    How to uninstall this plugin? I deleted all references, set the plugin to false in the config but still got a crash when restarting the project...

    Leave a comment:


  • replied
    Originally posted by Slayemin View Post
    Pardon my noob questions here, but I'm a bit confused on a few things.

    What I'm trying to do is use my own skeleton and bind its bones to the leap motion hands. I'm looking at the way it was implemented for the BasicLeapCharacter blueprint through the LeapAnimBodyConnector.

    The LeapAnimBodyConnector has this funky function called "ChangeBasis". It takes what looks like a pre-rotated value and does something with a post rotated value. I conceptually don't understand what's going on here and how it relates to the skel mesh of the character and the final output. For the left hand, the pre-rotation value is (-90,0,0) and the post rotation value is (0,0,90). How are you coming up with these values? What bones are you looking at?


    I'm also really new to animation and I've never seen the FABRIK and Two Bone IK stuff. I'm guessing that the ChangeBasis method is related to the inverse kinematics stuff, but I'm a bit lost. The FABRIK documentation isn't very helpful in explaining what it does conceptually.
    I'm glad you've asked the question, I was hoping someone would dig deep enough to ask about the implementation. The change basis function rotates all the AnimBody bones (that are parented, so if you rotate the body it will rotate all the bones) to a new basis. Pre-rotation applies input rotation before you're in the new space so it will affect all of your input rotations. In contrast post-rotation only rotates it by a constant amount and will not affect the basis of the input.

    The reason for doing all of this is that in component space (e.g. character blueprints) X is forward Y is right, Z is up; but in animation Y is forward, X is left, Z is up. So in order for everything to appear correct we need to change the rotation space to the new Y = Forward orientation. That is all that function does.

    Two-bone IK guesses a bone posture from a final position of the 'tip' bone. Typically you have another point where you encourage e.g. your elbow to bend towards. FABRIK on the other hand is a more general solver which can be have many more bones, so you can specify how long the bone chain is that gets affected by the 'tip' bone. In the leap rigging I currently have, I do not use Two-bone IK, instead we have only FABRIK nodes for spine/neck guessing for head positioning (from HMD) and shoulder guessing for your elbows. All other bones use the orientation values obtained from the leap motion. If we instead used IK for your hands, they would not always line up with what your real arms are doing because you would be disregarding the elbow and arm orientations that the leap motion sees for guessed positions. There are some minor problems with this approach, which boil down to mismatching arm and bone lengths, but for that you will always have some sort of compromise.

    If you're trying to use your own skeleton, it may be worthwhile to look at animation re-targeting. Down the line I have to come up with a cleaner solution for replacing skeletons/meshes.

    Originally posted by Spittygood View Post
    Getnamo,
    AWESOME job on the physics addition....solved what I wanted to do and was helpful in learning more about the engine. Have you tried the leap image hands for unity? That would be amazing to get into this plug-in!! The actual image of your hands instead of the twitchy fingers is a presence enhancer for sure.

    I am curious how hard you think this would be to implement.
    I will be testing these assets soon and their implementation are in the pipeline

    Leave a comment:

Working...
X