Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • replied
    Originally posted by Joe1029 View Post
    Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

    Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


    EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
    Update to 0.7.4
    -Normalized vectors such as palm normal were not properly adjusted for
    HMD view direction, this is has now been fixed.

    Updated the plugin because I realized that the normalized vectors which only used the convertLeapToUE function (such as the palm normal/direction) did not adjust for hmd use (topdown view needs to have it's axis rotated so that the perspective remains correct) this has been fixed.

    So with 0.7.4 you just need to use the above example I showed earlier and it will be correct even when you tick optimize for hmd.

    Edit:
    Update to 0.7.6
    -Added Raw Image events which give much cleaner support and memory
    efficient toggling of streaming images from the device.
    -Multiple components are now properly supported. Reason for error was
    mismanaged GC of on-demand UObject subclasses. Plugin will still emit
    multi-component warning for performance (once per play) reasons (redundancy/duplication).
    -All logging switched to LeapPluginLog sub-category

    The multiple component crash is now completely fixed. The reason for the delayed crashing was because garbage collection appeared to use 60 seconds as its interval, so the plugin was crashing because of mismanagement of my custom UObject references (created once and updated per need instead of recreating each frame for performance considerations), this has now been properly cleaned up.

    Decided the leap image polling api was a bit messy so added a convenient event for getting raw leap images

    it used to look like this


    you can still use that method but you can now do something much cleaner using events, such as


    both of which give the following billboard results (note the finger and palm debug visualization is a result of a separate graph)


    To receive the events and to enable image support in general call enable image suppport on your Leap Controller component


    if you use the images intermittently (e.g. fading real world in and out over the virtual one), you should toggle the image settings when they not in use to reclaim memory, performance, and help minimize usb bandwidth use.

    Example of a toggle
    Last edited by getnamo; 11-03-2014, 12:58 AM.

    Leave a comment:


  • replied
    Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

    Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


    EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
    Attached Files
    Last edited by Joe1029; 11-01-2014, 08:13 PM.

    Leave a comment:


  • replied
    Originally posted by machinimated View Post
    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
    Good catch. Currently multiple component use is undefined. I'm hoping to turn these into some sort of multi-cast in the future, which would allow you to have multiple components without issue, but for now just use one component


    Originally posted by mattplu View Post
    I'm hoping someone can offer some advice. I'm trying to create actual spheres instead of the debug spheres so they can collide with objects. I haven't found a way yet to create a static sphere and pick out the specific joint to attach each sphere to (most likely the best solution if possible).

    Instead I modified the Draw Debug Sphere function to spawn a blueprint actor with sphere instead and then set the lifespan to something extremely short. I also tried using an event tick to delete all actors of that blueprint class. It works for a while but eventually UE4 leaves a ghost of the deleted actor in the scene outliner and it crashes.

    Any one experience the same or found a way to create a simple set of spheres and move them rather than respawn (basically how to get the location of each and every joing LEAP provides).

    Thanks in advance!
    I missed this earlier in the thread, there are various ways to make this work. Remember one important rule in programming: creating and destroying is expensive, re-use where possible. One way would be to initially create the necessary spheres and then move/hide them based on use. I did something similar earlier, but beware the graphs are a bit complicated and not optimal (and tbh should be in C++), but as an example of drawing a 'fast sphere':



    and then hiding the ones no longer in use by a tick function like this



    BPList is an array of FastSphereBP (your sphere actor which can have collision). This is, again, not an ideal solution, but something I did to increase FPS for an oculus/hmd demo and shows one type of approach in getting this to work.

    Leave a comment:


  • replied
    Originally posted by machinimated View Post
    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
    Great!!! Is working now! Thanks machinimated!!!

    Leave a comment:


  • replied
    Just created a First Person Blueprint and added the Leap Event Interface to the Character with a Stabilized Palm Position pointing to a Debug Sphere.
    After 60 seconds it crashed.
    Attached Files

    Leave a comment:


  • replied
    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.

    Leave a comment:


  • replied
    Originally posted by Joe1029 View Post
    Hey getnamo since you're a programmer I assume you understand the leap API well enough. I've been looking at the best way to get the rotation data and give that to the hand bone. I've been using palm normal and direction but my understanding is that the hand bone rotates at the wrist while the leap is grabbing the rotation from the center of the palm, is there a way to correct this?
    NB: Rotation is location independent, you can easily use the palm orientation (and this is what you are looking for) and use it to rotate the wrist bone in your skeleton.

    Out of curiosity I took a look at this. Referring to the Leap API you can see that there are the two vectors reported that are relevant (as you mentioned). These are palmNormal() and direction(). Now one way you can get your orientation is by doing the following:
    Code:
        float pitch = hand.direction().pitch();
        float yaw = hand.direction().yaw();
        float roll = hand.palmNormal().roll();
    But since UE blueprints have convenience nodes, I find it easier to do the following



    Which is simply taking the normal and the direction and making a rotator from the Z and X axis, that's it.

    We can show this is correct by using a convenience debug function like this



    which gives the following result (along with debug showing finger positions for context)



    Also noticed that the palm normal was being scaled (which it shouldn't be) so I fixed that and updated the plugin to 0.7.3. This wouldn't affect the above graph however.

    Originally posted by fabianoberlim
    Tried in another computer and the engine still crashes. I notice that my actor (character) gets delete after 60 seconds. The engine does not crash until "Event Hand Moved" is activated.
    Maybe there's a UPROPERTY missing and the garbage collector is deleting the actor or component.
    Does anybody can replicate it?
    I have tried your example in the third person blueprint template and it isn't crashing for me, let's try to narrow this down. Make a new project, run me through which type of project (e.g. 3rd person template, blueprint) /etc so I can replicate it exactly. Also if you can send me your logs (found in {Project Root}/Saved/Logs), we should be able to identify what is going on.


    Originally posted by Luis Oliveira View Post
    Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

    "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

    Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.
    It is probably best that you start with a template character which will have all the movement input bindings conveniently set up. For really basic movement you can simply re-parent your pawn to Character.h.
    Last edited by getnamo; 10-31-2014, 10:10 AM.

    Leave a comment:


  • replied
    Tried in another computer and the engine still crashes. I notice that my actor (character) gets delete after 60 seconds. The engine does not crash until "Event Hand Moved" is activated.
    Maybe there's a UPROPERTY missing and the garbage collector is deleting the actor or component.
    Does anybody can replicate it?

    Leave a comment:


  • replied
    Hey getnamo since you're a programmer I assume you understand the leap API well enough. I've been looking at the best way to get the rotation data and give that to the hand bone. I've been using palm normal and direction but my understanding is that the hand bone rotates at the wrist while the leap is grabbing the rotation from the center of the palm, is there a way to correct this?

    Leave a comment:


  • replied
    I've been able to compile a new project. So I will test in my main project and see what happens.
    Originally posted by getnamo View Post
    Tested this in blueprint for both finger and hand (finger is a subclass of pointable, the only other class that has timeVisible code), it printed for longer than 60 seconds without issue, so I could not replicate this bug. The code is simply a pass-through for the underlying leap object, so it shouldn't be the reason for a bug. Are you referring to 'getFrame' instead which supports up to 60 frames (valid index 0-59)? Please elaborate what specific setup/steps cause this bug to show up?
    Leap still crashes with a very simple project. I have nothing but the character in the scene with this setup inside blueprints character.
    It crash both in the editor and the compiled project. It actually crash around 40 to 59 seconds.
    Click image for larger version

Name:	SimpleBlueprintsSetup.jpg
Views:	1
Size:	158.7 KB
ID:	1060410
    Last edited by fabianoberlim; 10-30-2014, 02:16 PM.

    Leave a comment:


  • replied
    Originally posted by Luis Oliveira View Post
    Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

    "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

    Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.
    Did you enable the leap blueprint props? Its at the top of your BP next to the compile and save button.

    Leave a comment:


  • replied
    Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

    "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

    Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.

    Leave a comment:


  • replied
    Originally posted by Luis Oliveira View Post
    Hm, that makes sense. I'll try it.

    Also I think this may have to do with my pawn blueprint. I don't know if I'm controlling it. I tried searching for pawn tutorials but no results. After I create pawn should I just drag and drop or do I need to do something with it?
    This really covers basics, so I recommend following unreal tutorials first. That said you need to make sure that the pawn is set as the default pawn in your game mode.

    Leave a comment:


  • replied
    Hm, that makes sense. I'll try it.

    Also I think this may have to do with my pawn blueprint. I don't know if I'm controlling it. I tried searching for pawn tutorials but no results. After I create pawn should I just drag and drop or do I need to do something with it?

    Leave a comment:


  • replied
    Originally posted by Luis Oliveira View Post
    I added through the Option B, and I even ticked all the HMD options because I'm using with the Oculus.
    But still, nothing happens.
    I recommend not using the rift first, but ensure it works by using the regular on-table placement and showing the debug hands. You may be simply not seeing the hand because it is off-screen. Understand that using Optimize for HMD will adjust the positions are rotations with the expectation that you have your leap mounted, essentially requiring you to put on the hmd to preview the hands.

    Leave a comment:

Working...
X