Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    #91
    Hm, that makes sense. I'll try it.

    Also I think this may have to do with my pawn blueprint. I don't know if I'm controlling it. I tried searching for pawn tutorials but no results. After I create pawn should I just drag and drop or do I need to do something with it?

    Comment


      #92
      Originally posted by Luis Oliveira View Post
      Hm, that makes sense. I'll try it.

      Also I think this may have to do with my pawn blueprint. I don't know if I'm controlling it. I tried searching for pawn tutorials but no results. After I create pawn should I just drag and drop or do I need to do something with it?
      This really covers basics, so I recommend following unreal tutorials first. That said you need to make sure that the pawn is set as the default pawn in your game mode.
      Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

      Comment


        #93
        Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

        "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

        Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.

        Comment


          #94
          Originally posted by Luis Oliveira View Post
          Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

          "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

          Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.
          Did you enable the leap blueprint props? Its at the top of your BP next to the compile and save button.

          Comment


            #95
            I've been able to compile a new project. So I will test in my main project and see what happens.
            Originally posted by getnamo View Post
            Tested this in blueprint for both finger and hand (finger is a subclass of pointable, the only other class that has timeVisible code), it printed for longer than 60 seconds without issue, so I could not replicate this bug. The code is simply a pass-through for the underlying leap object, so it shouldn't be the reason for a bug. Are you referring to 'getFrame' instead which supports up to 60 frames (valid index 0-59)? Please elaborate what specific setup/steps cause this bug to show up?
            Leap still crashes with a very simple project. I have nothing but the character in the scene with this setup inside blueprints character.
            It crash both in the editor and the compiled project. It actually crash around 40 to 59 seconds.
            Click image for larger version

Name:	SimpleBlueprintsSetup.jpg
Views:	1
Size:	158.7 KB
ID:	1060410
            Last edited by fabianoberlim; 10-30-2014, 02:16 PM.

            Comment


              #96
              Hey getnamo since you're a programmer I assume you understand the leap API well enough. I've been looking at the best way to get the rotation data and give that to the hand bone. I've been using palm normal and direction but my understanding is that the hand bone rotates at the wrist while the leap is grabbing the rotation from the center of the palm, is there a way to correct this?

              Comment


                #97
                Tried in another computer and the engine still crashes. I notice that my actor (character) gets delete after 60 seconds. The engine does not crash until "Event Hand Moved" is activated.
                Maybe there's a UPROPERTY missing and the garbage collector is deleting the actor or component.
                Does anybody can replicate it?

                Comment


                  #98
                  Originally posted by Joe1029 View Post
                  Hey getnamo since you're a programmer I assume you understand the leap API well enough. I've been looking at the best way to get the rotation data and give that to the hand bone. I've been using palm normal and direction but my understanding is that the hand bone rotates at the wrist while the leap is grabbing the rotation from the center of the palm, is there a way to correct this?
                  NB: Rotation is location independent, you can easily use the palm orientation (and this is what you are looking for) and use it to rotate the wrist bone in your skeleton.

                  Out of curiosity I took a look at this. Referring to the Leap API you can see that there are the two vectors reported that are relevant (as you mentioned). These are palmNormal() and direction(). Now one way you can get your orientation is by doing the following:
                  Code:
                      float pitch = hand.direction().pitch();
                      float yaw = hand.direction().yaw();
                      float roll = hand.palmNormal().roll();
                  But since UE blueprints have convenience nodes, I find it easier to do the following



                  Which is simply taking the normal and the direction and making a rotator from the Z and X axis, that's it.

                  We can show this is correct by using a convenience debug function like this



                  which gives the following result (along with debug showing finger positions for context)



                  Also noticed that the palm normal was being scaled (which it shouldn't be) so I fixed that and updated the plugin to 0.7.3. This wouldn't affect the above graph however.

                  Originally posted by fabianoberlim
                  Tried in another computer and the engine still crashes. I notice that my actor (character) gets delete after 60 seconds. The engine does not crash until "Event Hand Moved" is activated.
                  Maybe there's a UPROPERTY missing and the garbage collector is deleting the actor or component.
                  Does anybody can replicate it?
                  I have tried your example in the third person blueprint template and it isn't crashing for me, let's try to narrow this down. Make a new project, run me through which type of project (e.g. 3rd person template, blueprint) /etc so I can replicate it exactly. Also if you can send me your logs (found in {Project Root}/Saved/Logs), we should be able to identify what is going on.


                  Originally posted by Luis Oliveira View Post
                  Heh, thanks but I've been working with UE4 for some time now (but mainly just making level design). I know the basics except for the classes and blueprints part, I can only make openable doors lol.

                  "That said you need to make sure that the pawn is set as the default pawn in your game mode." - Yeah I did that but it doesn't really affect my level. I can't move anymore after I do that, but maybe the player controls are inside the default pawn. I still have a lot to learn ^^

                  Do i need to add stuff to the components part? Because there's nothing there right now. I tried adding a camera but that didn't help much.
                  It is probably best that you start with a template character which will have all the movement input bindings conveniently set up. For really basic movement you can simply re-parent your pawn to Character.h.
                  Last edited by getnamo; 10-31-2014, 10:10 AM.
                  Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                  Comment


                    #99
                    I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.

                    Comment


                      Just created a First Person Blueprint and added the Leap Event Interface to the Character with a Stabilized Palm Position pointing to a Debug Sphere.
                      After 60 seconds it crashed.
                      Attached Files

                      Comment


                        Originally posted by machinimated View Post
                        I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
                        Great!!! Is working now! Thanks machinimated!!!

                        Comment


                          Originally posted by machinimated View Post
                          I was working through this thread last night and encountered the same crash after 60 seconds that some of you are getting. It turns out that when following the instructions at the top of the thread (step 5), I had added the leap component in both the component tab AND in the event graph, instead of just doing one or the other. Simply removing the leap component from the component tab fixed the crash for me.
                          Good catch. Currently multiple component use is undefined. I'm hoping to turn these into some sort of multi-cast in the future, which would allow you to have multiple components without issue, but for now just use one component


                          Originally posted by mattplu View Post
                          I'm hoping someone can offer some advice. I'm trying to create actual spheres instead of the debug spheres so they can collide with objects. I haven't found a way yet to create a static sphere and pick out the specific joint to attach each sphere to (most likely the best solution if possible).

                          Instead I modified the Draw Debug Sphere function to spawn a blueprint actor with sphere instead and then set the lifespan to something extremely short. I also tried using an event tick to delete all actors of that blueprint class. It works for a while but eventually UE4 leaves a ghost of the deleted actor in the scene outliner and it crashes.

                          Any one experience the same or found a way to create a simple set of spheres and move them rather than respawn (basically how to get the location of each and every joing LEAP provides).

                          Thanks in advance!
                          I missed this earlier in the thread, there are various ways to make this work. Remember one important rule in programming: creating and destroying is expensive, re-use where possible. One way would be to initially create the necessary spheres and then move/hide them based on use. I did something similar earlier, but beware the graphs are a bit complicated and not optimal (and tbh should be in C++), but as an example of drawing a 'fast sphere':



                          and then hiding the ones no longer in use by a tick function like this



                          BPList is an array of FastSphereBP (your sphere actor which can have collision). This is, again, not an ideal solution, but something I did to increase FPS for an oculus/hmd demo and shows one type of approach in getting this to work.
                          Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                          Comment


                            Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

                            Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


                            EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
                            Attached Files
                            Last edited by Joe1029; 11-01-2014, 08:13 PM.

                            Comment


                              Originally posted by Joe1029 View Post
                              Hey Getnamo I'm still working on the wrist rotation I almost have it set up right but the last issue is that when I turn my hand towards me or away from me (roll) the other axis flip, if hand is facing me the yaw flips but pitch works fine, if I turn the hand away from me the pitch flips and the yaw works. Is there a way to keep the orientation of the hand so when the hand pitches foward it would be based on the orientation rather than the raw numbers?

                              Here is my set up so far. I tried implementing the combine rotators you mentioned earlier and am looking at the hydra forum but I don't know what is the other input I use for combine rotator.


                              EDIT: I changed the float value in the yaw > pitch to positive 1 which made my hand work properly when it is facing towards me but completely flips my yaw and pitch when facing away from me. Still trying to find a way to get an orientation so the rotation knows which was is positive and which way is negative. Also I do have Gimbal lock happening so I do need that combine rotator in my BP
                              Update to 0.7.4
                              -Normalized vectors such as palm normal were not properly adjusted for
                              HMD view direction, this is has now been fixed.

                              Updated the plugin because I realized that the normalized vectors which only used the convertLeapToUE function (such as the palm normal/direction) did not adjust for hmd use (topdown view needs to have it's axis rotated so that the perspective remains correct) this has been fixed.

                              So with 0.7.4 you just need to use the above example I showed earlier and it will be correct even when you tick optimize for hmd.

                              Edit:
                              Update to 0.7.6
                              -Added Raw Image events which give much cleaner support and memory
                              efficient toggling of streaming images from the device.
                              -Multiple components are now properly supported. Reason for error was
                              mismanaged GC of on-demand UObject subclasses. Plugin will still emit
                              multi-component warning for performance (once per play) reasons (redundancy/duplication).
                              -All logging switched to LeapPluginLog sub-category

                              The multiple component crash is now completely fixed. The reason for the delayed crashing was because garbage collection appeared to use 60 seconds as its interval, so the plugin was crashing because of mismanagement of my custom UObject references (created once and updated per need instead of recreating each frame for performance considerations), this has now been properly cleaned up.

                              Decided the leap image polling api was a bit messy so added a convenient event for getting raw leap images

                              it used to look like this


                              you can still use that method but you can now do something much cleaner using events, such as


                              both of which give the following billboard results (note the finger and palm debug visualization is a result of a separate graph)


                              To receive the events and to enable image support in general call enable image suppport on your Leap Controller component


                              if you use the images intermittently (e.g. fading real world in and out over the virtual one), you should toggle the image settings when they not in use to reclaim memory, performance, and help minimize usb bandwidth use.

                              Example of a toggle
                              Last edited by getnamo; 11-03-2014, 12:58 AM.
                              Plugins: TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

                              Comment


                                Hey Getnamo the axis flipping is still happening. I updated the plug in to 7.6 and I still have the same problem. Any suggestions?
                                Last edited by Joe1029; 11-03-2014, 08:55 PM.

                                Comment

                                Working...
                                X