Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Hey Guys I'm using this plugin in ue 4.10 and whenever I play my game my hands are swapped. My leap motion thinks i'm in desktop mode or something. I Have tried chaning this setting everywhere. Has anyone else experienced this or know a possible solution?

    Leave a comment:


  • replied
    Originally posted by Sycol View Post
    Hi, I'm the guy with the axis problem. I'm new with leap and with Ue4 so im very lost. I want to do something like this https://www.youtube.com/watch?v=am6Rt2toZZc
    but it looks like that: https://youtu.be/3apQNG_30qE [I have not much idea of what i'm doing] so i want to ask you if you can please, please show me how?
    Thanks for all.
    From your video it seems you're using the official plugin, if you're using the unofficial plugin you can use the palm orientation value to drive the desires behavior.

    Originally posted by vimaxus View Post
    I think the low latency feeling only comes from the "temporal warping bias" that is presented as an option. If I put it to 0 I see the same "lag" as in UE4. I thought about doing something like this for some time and now I have the motivation as it makes quite a difference.
    I plan to get the relative position from hands to headset and move them to where they were relative to the HMD x ms before.
    How can I get to do that at each update hands event so that it doesn't add-up.

    I know it's not very inteligible, I'll try to add a blueprint or some scanned sketches.
    Yep leap uses timewarp for their unity assets, its on the to do list but may take some time before I get to it. What they do is take a time sample along with the position and then grab the hmd orientation at the time. Then during late update they check the hmd orientation change between now and the timesample that was obtained earlier. The rotation difference is what is applied to make the timewarp.

    If you manage to get it added make a pull request

    Originally posted by Ders92 View Post
    Hello!

    I'm currently using the getnamo plugin for leap motion in unreal engine 4 in combination with occulus rift dk2 to make a game. The game will be played on a virtual table so the players should be able to reach everywhere on the map (it should feel like you are sitting around a real table). In order to achieve this I want to change the "world to meters" parameter in world settings under VR, but when i increase it to for example 200 (default is 100) the leap motion hands start behaving weird. A cross eye effect occurs and if I increase it even more the leap hands eventually disappears completely. One way to solve it is to keep the world to meters parameter as it is and just scale the game world down, but this resulted in issues with the navigation mesh as I had to scale everything down a lot.

    So my question is if this problem when increasing the "world to meters" parameter is at all connected to the leap motion plugin or is it some occulus/ue4 specific problem? Or am I missing something obvious here? Haven't found anyone with the same problem so im starting to suspect that it might be something fundamental im missing. It's my first time working with ue4, occulus and leap motion so everything is kind of new.
    Not sure about this one, first time I've seen this issue. You can try to correct world to meters using ipd instead (stereo e=<ipd value here>)

    Originally posted by dmorgan View Post
    what the easiest way to handle picking up an object with the leap controller? How to get collision events per hand? Thanks
    Made a video series on this topic, follow along to learn how to make e.g. a jenga game:

    Part 1


    Part 2


    Part 3
    Last edited by getnamo; 02-29-2016, 10:51 AM.

    Leave a comment:


  • replied
    what the easiest way to handle picking up an object with the leap controller? How to get collision events per hand? Thanks

    Leave a comment:


  • replied
    Thanks! I don't know I'm doing so I really appreciate the response.

    Leave a comment:


  • replied
    Hello!

    I'm currently using the getnamo plugin for leap motion in unreal engine 4 in combination with occulus rift dk2 to make a game. The game will be played on a virtual table so the players should be able to reach everywhere on the map (it should feel like you are sitting around a real table). In order to achieve this I want to change the "world to meters" parameter in world settings under VR, but when i increase it to for example 200 (default is 100) the leap motion hands start behaving weird. A cross eye effect occurs and if I increase it even more the leap hands eventually disappears completely. One way to solve it is to keep the world to meters parameter as it is and just scale the game world down, but this resulted in issues with the navigation mesh as I had to scale everything down a lot.

    So my question is if this problem when increasing the "world to meters" parameter is at all connected to the leap motion plugin or is it some occulus/ue4 specific problem? Or am I missing something obvious here? Haven't found anyone with the same problem so im starting to suspect that it might be something fundamental im missing. It's my first time working with ue4, occulus and leap motion so everything is kind of new.

    Leave a comment:


  • replied
    I think the low latency feeling only comes from the "temporal warping bias" that is presented as an option. If I put it to 0 I see the same "lag" as in UE4. I thought about doing something like this for some time and now I have the motivation as it makes quite a difference.
    I plan to get the relative position from hands to headset and move them to where they were relative to the HMD x ms before.
    How can I get to do that at each update hands event so that it doesn't add-up.

    I know it's not very inteligible, I'll try to add a blueprint or some scanned sketches.

    Leave a comment:


  • replied
    Hi, I'm the guy with the axis problem. I'm new with leap and with Ue4 so im very lost. I want to do something like this https://www.youtube.com/watch?v=am6Rt2toZZc
    but it looks like that: https://youtu.be/3apQNG_30qE [I have not much idea of what i'm doing] so i want to ask you if you can please, please show me how?
    Thanks for all.

    Leave a comment:


  • replied
    Originally posted by Rend View Post
    @Getnamo
    Does this work with the Orion SDK?
    Yep just update your runtime. The only thing the plugin won't have is the low latency update from closer to metal api, will need to wait for an api update before I can have a look at it.

    Leave a comment:


  • replied
    @Getnamo
    Does this work with the Orion SDK?

    Leave a comment:


  • replied
    Originally posted by Deams View Post
    We might see such an interface happen in the near future.
    There's quite a lot going on around VR, and I understand that teams like Epic have other priorities than body rigging for now.
    But it could make quite an interesting project during my thesis if I still have time after the original topic.

    Have you had the time to take a look at how I took care of the fingers bones?
    Code is still a bit messy since it was my first Unreal Engine coding session, but I could clean it up and comment more seriously if you have any use for it. Same for the general c++ implementation, if it could somehow be useful, we can discuss the architecture to integrate it to the master.
    I like the way you handle as much as possible in C++ and your overall structure is good. I think C++ anim blueprint nodes are the way to go allowing for cleaner setups downstream if we can handle input diversity well (orientation only FK (e.g. multiple IMUs) vs positional tracking IK (e.g. motion controllers)). The plugin as it is, is due for a refactoring and I would love to incorporate these changes as well start the Body State separation so all input plugins could target the same animation chain. I'll pm you if you're interested in helping/carrying this further.

    The end goal in my mind would be to allow a developer to select a skeletal mesh with humanoid like rig and just select e.g. map Body State to this skeleton and have the plugin do all the hard work of re-targeting if necessary and switch animation chain to handle the type data available (do we have orientation only? Hands only or full body? Mixed inputs? which one is more accurate for this frame?).


    Originally posted by Deams View Post
    By the way, have you played with the "image-hand" that Leap integrated to their unity3D asset?
    I didn't see it in their UE4 plugin.
    I have and this plugin's UE version is in the works as we speak. It's fairly close to being ready, but there are quite a few changes to get right and given the fully deferred renderer of UE4, it may not look exactly the same. One of the non obvious aspects of image hands is that you have to use the leap IPD (40mm) for the hand look and interact with the world as you would expect which makes everything feel a bit bigger.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    Just had a look at the commit, that's a beefy commit, great work!

    I should have a chance to play with it this weekend, always wanted a better understanding of setting up IK/animation in c++ and I think this is a great resource to brush up on that department.


    Long term thoughts:
    In the end I think convenience setups such as rigged characters should be abstracted away from the hardware plugins themselves. A full 'abstract skeleton' target could be used that can take input from multiple devices and merge the information by the best data available. From that skeleton you would then extract the information and forward it to any output node e.g. a convenience rigged character. Then whether you use just floating rigged hands, poly models or a full body character you would have the same backend and you wouldn't need to change your rigging for new hardware inputs and you would now support mixed inputs. In my mind this is something akin to a Body State Plugin. You could further expand this by adding some smart class to handle skeleton re-targeting, which could automatically match various skeletal meshes to the abstract one and perhaps allow for almost a one click change to swap any rigged character to support a new skeletal mesh you want to replicate your movement. That would be the ideal setup in my mind.

    The hard part of this is figuring out a way in which plugins such as this one can forward input automatically to a target that may not be there (unless something like this idea makes it into the engine and we can depend on it existing).
    We might see such an interface happen in the near future.
    There's quite a lot going on around VR, and I understand that teams like Epic have other priorities than body rigging for now.
    But it could make quite an interesting project during my thesis if I still have time after the original topic.

    Have you had the time to take a look at how I took care of the fingers bones?
    Code is still a bit messy since it was my first Unreal Engine coding session, but I could clean it up and comment more seriously if you have any use for it. Same for the general c++ implementation, if it could somehow be useful, we can discuss the architecture to integrate it to the master.

    By the way, have you played with the "image-hand" that Leap integrated to their unity3D asset?
    I didn't see it in their UE4 plugin.

    Leave a comment:


  • replied
    Originally posted by Deams View Post
    Finally had a bit of time between the exams to work on this.


    It seems that the only problem there for the leap motion plugin was the lack of the export macro on the classes or specific methods.
    Might be the same for the Zip plugin.



    It seems that best practices are fuzzy for many people on many topics...
    Hopefully, more will pop up in the documentation.

    In the meantime, I decided to keep going into the conversion to C++.
    In short, it works but needs to be finished.
    Most of the blueprints functionalities are now also available from C++ but can also be manipulated via blueprints. This means that you can create a game mode using directly the c++ LeapCharacter class in the settings. The VRController will come later.
    The only blueprints really necessary to conform with the "best practices" is the animation blueprint.
    And since I'm lazy, I preferred adding custom animation nodes than connecting each bones of each fingers to all the nodes... I have to say, I'm impressed by your patience to make it even all look nice in there. :P

    More information in the commit message: Commit details

    There's still some work to do, but it should allow both the blueprint and c++ ways to coexist at each level.
    Just had a look at the commit, that's a beefy commit, great work!

    I should have a chance to play with it this weekend, always wanted a better understanding of setting up IK/animation in c++ and I think this is a great resource to brush up on that department.


    Long term thoughts:
    In the end I think convenience setups such as rigged characters should be abstracted away from the hardware plugins themselves. A full 'abstract skeleton' target could be used that can take input from multiple devices and merge the information by the best data available. From that skeleton you would then extract the information and forward it to any output node e.g. a convenience rigged character. Then whether you use just floating rigged hands, poly models or a full body character you would have the same backend and you wouldn't need to change your rigging for new hardware inputs and you would now support mixed inputs. In my mind this is something akin to a Body State Plugin. You could further expand this by adding some smart class to handle skeleton re-targeting, which could automatically match various skeletal meshes to the abstract one and perhaps allow for almost a one click change to swap any rigged character to support a new skeletal mesh you want to replicate your movement. That would be the ideal setup in my mind.

    The hard part of this is figuring out a way in which plugins such as this one can forward input automatically to a target that may not be there (unless something like this idea makes it into the engine and we can depend on it existing).

    Leave a comment:


  • replied
    Finally had a bit of time between the exams to work on this.

    Originally posted by getnamo View Post
    I haven't touched C++ project binding to plugins in a while, and I've just recently hit the same problem actually with my zip plugin!
    It seems that the only problem there for the leap motion plugin was the lack of the export macro on the classes or specific methods.
    Might be the same for the Zip plugin.

    Originally posted by getnamo View Post
    I'm going to try your fixes and get back to you, but thanks a lot for the suggestions! If this compiles for you without problems, consider making a pull request to have the changes added to the master branch.

    Edit: Back in 4.0 I would copy the class implementation files for editor (where plugins are isolated in their own dlls) and remove them in packaged solutions (your dlls are collapsed into a monolithic build so it would link without issue). So this is much cleaner than that hack.

    According to plugin documentation what we're doing is essentially creating a hard bind, but I believe it falls under

    If you are creating a game plugin (not engine plugin), and you want to allow the game to statically link against one of your plugin modules. This breaks the concept of a plugin a little bit, but is often useful for plugins that want to declare new UObject types that game classes can inherit from or use directly. The engine itself has no dependencies on this plugin, but the game project code and content may very well have direct dependencies.

    I'm not 100% sure on best practices, but it does seem to be the most convenient and when you want to use leap motion specific methods in C++ there really isn't a practical way to not use this approach.
    It seems that best practices are fuzzy for many people on many topics...
    Hopefully, more will pop up in the documentation.

    In the meantime, I decided to keep going into the conversion to C++.
    In short, it works but needs to be finished.
    Most of the blueprints functionalities are now also available from C++ but can also be manipulated via blueprints. This means that you can create a game mode using directly the c++ LeapCharacter class in the settings. The VRController will come later.
    The only blueprints really necessary to conform with the "best practices" is the animation blueprint.
    And since I'm lazy, I preferred adding custom animation nodes than connecting each bones of each fingers to all the nodes... I have to say, I'm impressed by your patience to make it even all look nice in there. :P

    More information in the commit message: Commit details

    There's still some work to do, but it should allow both the blueprint and c++ ways to coexist at each level.

    Leave a comment:


  • replied
    Originally posted by Deams View Post

    ...

    Is it intended?
    I haven't touched C++ project binding to plugins in a while, and I've just recently hit the same problem actually with my zip plugin!

    I'm going to try your fixes and get back to you, but thanks a lot for the suggestions! If this compiles for you without problems, consider making a pull request to have the changes added to the master branch.


    Edit: Back in 4.0 I would copy the class implementation files for editor (where plugins are isolated in their own dlls) and remove them in packaged solutions (your dlls are collapsed into a monolithic build so it would link without issue). So this is much cleaner than that hack.

    According to plugin documentation what we're doing is essentially creating a hard bind, but I believe it falls under

    If you are creating a game plugin (not engine plugin), and you want to allow the game to statically link against one of your plugin modules. This breaks the concept of a plugin a little bit, but is often useful for plugins that want to declare new UObject types that game classes can inherit from or use directly. The engine itself has no dependencies on this plugin, but the game project code and content may very well have direct dependencies.

    I'm not 100% sure on best practices, but it does seem to be the most convenient and when you want to use leap motion specific methods in C++ there really isn't a practical way to not use this approach.
    Last edited by getnamo; 01-08-2016, 04:57 PM.

    Leave a comment:


  • replied
    Originally posted by getnamo View Post
    1) Attach a component to your pawn/actor.

    2) Use the LeapEventInterface via multiple-inheritance
    I tried to add the LeapController component to a default character, but so far I can't managed to link the code.

    In the Char.h I include "LeapController.h" and define a pointer to the component.
    Code:
    class ULeapController* LeapController;
    In the Char.cpp, I instantiate the LeapController in the class constructor
    Code:
    ALeapCharacter::ALeapCharacter()
    {
    	PrimaryActorTick.bCanEverTick = true;
    
    	LeapController = CreateDefaultSubobject<ULeapController>(TEXT("LeapControllerComponent"));
    }
    But end up getting the following link error:
    Code:
    LeapCharacter.cpp.obj : error LNK2019: unresolved external symbol "private: static class UClass * __cdecl ULeapController::GetPrivateStaticClass(wchar_t const *)" (?GetPrivateStaticClass@ULeapController@@CAPEAVUClass@@PEB_W@Z) referenced in function "public: class ULeapController * __cdecl UObject::CreateDefaultSubobject<class ULeapController>(class FName,bool)" (??$CreateDefaultSubobject@VULeapController@@@UObject@@QEAAPEAVULeapController@@VFName@@_N@Z)
    Is there another step I'm forgetting about?

    Edit:
    It seems that adding the MinimalAPI tag for the ULeapController class fixes the compilation error to instantiate the component.
    Code:
    UCLASS(MinimalAPI, ClassGroup=Input, meta=(BlueprintSpawnableComponent))
    class ULeapController : public UActorComponent
    But the same kind of linking error happen when using a method of ULeapController.
    Example when using the EnableBackgroundTracking:
    Code:
    LeapCharacter.cpp.obj : error LNK2019: unresolved external symbol "public: void __cdecl ULeapController::EnableBackgroundTracking(bool)" (?EnableBackgroundTracking@ULeapController@@QEAAX_N@Z) referenced in function "public: __cdecl ALeapCharacter::ALeapCharacter(void)" (??0ALeapCharacter@@QEAA@XZ)
    Using the LEAPMOTION_API tag for the class definition of ULeapController does seem to export all methods.
    Code:
    class LEAPMOTION_API ULeapController : public UActorComponent
    Is it intended?
    Last edited by Deams; 01-08-2016, 04:42 PM. Reason: New data

    Leave a comment:

Working...
X