Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Hi all,


    I have been experimenting with the latest Leap Motion SDK, and the VR widgets in Unity.

    Here is the link:

    http://blog.leapmotion.com/introduci...lements-unity/



    Here is a video I recorded in Youtube:


    Hope we could see this kind of things in UE4 soon with Leap Motion. What do you think Getnamo??


    Cheers.

    Comment


      Hi All,

      I am using a blueprint with Pawn as its parent class. I observe that the hand can be moved over a very limited space in the viewport. What is wrong? How to make the movement cover the whole viewport? Please help.

      Comment


        Originally posted by mankindforward View Post
        Yes, I'm doing this on two separate computers. Two of everything, connecting over LAN. Movements are replicated properly, meaning Player 1 can move and Player 2 sees the him move. I'll keep digging but it sounds like this *should* work. Have you tested this scenario yourself? Or anybody else tried this? Perhaps I'm missing some configuration.
        Interesting. I do not have 2 leaps to replicate what you're doing, but based on my experience with networking you can choose specifically what should replicate and what shouldn't. You may be doing full actor replication instead of variable replication.

        If you use variable replication with hand/finger positions (e.g. AnimBody) then downstream they should affect the 2nd client only as expected. There are various approaches to what information gets updated over a network. A good primer on blueprint networking can be found here: https://www.youtube.com/watch?v=TbaOyvWfJE0

        Originally posted by Rasa View Post
        Needed your help how could I convert that mouse pointer to down direction and also I don't want camera to follow that mouse pointer
        You probably want to do an raycast originating from the hand's position in the negative Z axis. This would achieve what you're looking for

        Originally posted by Rasa View Post
        How could we do that Hand alone[Without character attached] for Leap motion.
        Already answered earlier.


        Originally posted by aretoon View Post
        Hey Getnamo,

        I just wanted to see if you could give me some direction on what I'm trying to do with your plug in. I want to assign custom functions to basic gestures. I.E. Swiping and rotating an object, pinching to zoom, or controlling the translation of an object by forming a fist and moving it. The goal is to utlize leap motion in showcasing products and having customers interact with it.

        I don't know if there are readily available sources for this, and as a beginner, I sometimes get extremely confused. How should I go about doing this? Where do I start experimenting? Which blueprint are the edits supposed to be done under? If you could give me your opinion as to how to do something similar and tell me the logic behind it I would really be grateful.

        Amazing job as always!
        Cheers,
        Artun
        You probably want to use the Swipe Gesture recognizer in a similar manner to



        See the github repo for more detailed documentation.

        Originally posted by Beunn View Post
        Hey Getnamo,

        Thanks for this amazing plugin !

        Can you tell me how to use it with C++ code ? (Listener with events, Controller...)

        Cheers,
        Beunn
        It would be similar to how its shown in the hydra plugin video with the main difference that you would add a leap component instead of a delegate subclass (leap plugin doesn't have this format) and you would set the interface to receive events via https://wiki.unrealengine.com/Interfaces_in_C%2B%2B.

        Originally posted by iparra View Post
        Hi all,

        I have been experimenting with the latest Leap Motion SDK, and the VR widgets in Unity.

        Here is the link:

        http://blog.leapmotion.com/introduci...lements-unity/

        Here is a video I recorded in Youtube: ...

        Hope we could see this kind of things in UE4 soon with Leap Motion. What do you think Getnamo??

        Cheers.
        These are great things, nothing is stopping you all from adding these kinds of widgets to the plugin, looking forward to seeing these things being tried.

        Originally posted by EVRTOP View Post
        How to put the reality of hand display
        The Passthrough effect seen iparra's video is being worked on however it is not yet ready for the plugin (need to solve 32bit textures first).

        Originally posted by Ittiam View Post
        Hi All,

        I am using a blueprint with Pawn as its parent class. I observe that the hand can be moved over a very limited space in the viewport. What is wrong? How to make the movement cover the whole viewport? Please help.
        See this post where we discuss how to change the origin of the leap in default mode.
        Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

        Comment


          Hi All,

          I'm wondering if anyone else is having a problem with gestures? I'm using "Event Hand Grabbed" and it seems like the leap only registers the gesture about half of the time, and only if the hands are located in a very specific location relative to the leap. I'm not sure if it's the leap itself, or something specific to my project or this plugin. Does anybody else have this problem, or know of a fix for it?

          Thanks,
          Pat

          Comment


            some progress in the finger rotation fix?
            pd: excuse my english.

            lfw/paid modeling, painting, texturing.

            Comment


              Hi there, just try your plug-in and it was awesome!! By the way I've tried with Oculus VR and something get weird. Hands&arms are reversed display (I move right hand, the left one in VR move). Also if I want to use only the arm detecting (freezing fingers/hands) will I be able to do it with your plug-in?

              P.S. I'm really new to UE, please spare me if I know too little, Thank you.

              Comment


                Originally posted by Tony Tong View Post
                Hi there, just try your plug-in and it was awesome!! By the way I've tried with Oculus VR and something get weird. Hands&arms are reversed display (I move right hand, the left one in VR move). Also if I want to use only the arm detecting (freezing fingers/hands) will I be able to do it with your plug-in?

                P.S. I'm really new to UE, please spare me if I know too little, Thank you.
                If you're using the rigged content, then hands reversal may be due to initial wrong guess by the leap software. Hide and show your hands again and it will typically get them right. If you're using a custom approach on the other hand, remember that you need to enable HMD mode (Optimize for HMD and auto-shift and auto-rotate) in order for the hands to appear correctly. Rigged content automatically adjusts this.

                Regarding arm detection, you can query Arm location from the 'Hand Moved' event and only use that to track your characters movements.

                Originally posted by knack View Post
                some progress in the finger rotation fix?
                Not yet, will take a look at this when I get some time :/

                Originally posted by ppaolell View Post
                Hi All,

                I'm wondering if anyone else is having a problem with gestures? I'm using "Event Hand Grabbed" and it seems like the leap only registers the gesture about half of the time, and only if the hands are located in a very specific location relative to the leap. I'm not sure if it's the leap itself, or something specific to my project or this plugin. Does anybody else have this problem, or know of a fix for it?

                Thanks,
                Pat
                Leap software may have issues detecting a grabbing gesture when your hand is facing away. If you're finding a specific location causes a problem, try to make a video of it and it will be easier to find the root cause.
                Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                Comment


                  Hi, I'm trying to use it now, and I just downloaded newest version of it, I also successfully see my hands on the screen, however hands on screen is flipped, for example, my thumb movement reflect on the little finger at the screen.
                  It would be helpful if I know what reason could make this is happening!

                  Comment


                    this plugin is incredible and works great, thanks so much for this integration work!
                    Earthborn Interactive | Flutter Bombs | MageWorks

                    Comment


                      Originally posted by Harry.Kim View Post
                      Hi, I'm trying to use it now, and I just downloaded newest version of it, I also successfully see my hands on the screen, however hands on screen is flipped, for example, my thumb movement reflect on the little finger at the screen.
                      It would be helpful if I know what reason could make this is happening!
                      I found what was the problem. It's related to VR mode. https://developer.leapmotion.com/vr

                      Comment


                        Hi Getnamo,

                        In this plugin (and your Myo plugin and your Hydra plugin) you make use of a "delegate" concept. However, looking through your code you do not seem to use any of UE4's delegate system. Did you look at that and, if so, why did you choose your own custom delegate implementation (mostly through subclassing and virtual functions)?

                        I ask because I'm making my own plugin now and I wonder if I should use UE4's system.

                        Comment


                          when i'm retrieving a world-space value for the distal position of the index finger (or any finger segment position for that matter), i'm storing this value, and converting it to screen space. this has been extremely accurate in windowed mode, however in full screen mode, there seems to be an offset when grabbing this value and converting it. any idea if this is related to the plugin code? i'm trying to pinpoint various locations to check. does full screen enable hmd offset compensation even when there isn't any hmd device enabled?
                          Earthborn Interactive | Flutter Bombs | MageWorks

                          Comment


                            Originally posted by Harry.Kim View Post
                            I found what was the problem. It's related to VR mode. https://developer.leapmotion.com/vr
                            Was it a toggle you forgot to use?

                            Originally posted by paradoc View Post
                            this plugin is incredible and works great, thanks so much for this integration work!
                            Glad you found it useful !

                            Originally posted by paradoc View Post
                            when i'm retrieving a world-space value for the distal position of the index finger (or any finger segment position for that matter), i'm storing this value, and converting it to screen space. this has been extremely accurate in windowed mode, however in full screen mode, there seems to be an offset when grabbing this value and converting it. any idea if this is related to the plugin code? i'm trying to pinpoint various locations to check. does full screen enable hmd offset compensation even when there isn't any hmd device enabled?
                            I believe HMD mode automatically adds the leap-rift offset, which is 8cm FVector(8,0,0) rotated by the HMD. This will only be added if you enable HMD mode.

                            Originally posted by bullale View Post
                            Hi Getnamo,

                            In this plugin (and your Myo plugin and your Hydra plugin) you make use of a "delegate" concept. However, looking through your code you do not seem to use any of UE4's delegate system. Did you look at that and, if so, why did you choose your own custom delegate implementation (mostly through subclassing and virtual functions)?

                            I ask because I'm making my own plugin now and I wonder if I should use UE4's system.
                            Back when I started with the hydra plugin (4.0) there wasn't as much documentation so I wasn't aware of the UE4 delegate system. I was looking for an architecture that would allow me to add event driven, auto-ticking to any class where you would've wanted it in C++. As time went on and I learned more about UE4 it became clear that components offer more flexibility and you could receive events by adding an interface. This allowed each plugin to now extend functionality directly in blueprints rather than C++. I still believe this, along with including convenience characters/actors that can be used for common use cases (e.g. rigged character), is the best approach.

                            If you can see a way to make your code cleaner by using the UE delegate system, use it. Let us know how it goes!
                            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                            Comment


                              Originally posted by getnamo View Post
                              Back when I started with the hydra plugin (4.0) there wasn't as much documentation so I wasn't aware of the UE4 delegate system. I was looking for an architecture that would allow me to add event driven, auto-ticking to any class where you would've wanted it in C++. As time went on and I learned more about UE4 it became clear that components offer more flexibility and you could receive events by adding an interface. This allowed each plugin to now extend functionality directly in blueprints rather than C++. I still believe this, along with including convenience characters/actors that can be used for common use cases (e.g. rigged character), is the best approach.

                              If you can see a way to make your code cleaner by using the UE delegate system, use it. Let us know how it goes!
                              Thank you for your reply.

                              One major advantage of the UE delegate system is that you can have more than one listener. I think this could still work with the UActorComponent. It might be possible to have many Actors with LeapController within. Anyway, the documentation is still not quite there yet so I don't fully get it and will probably not use it.

                              I have another question for you but it's not Leap-specific so I will ask by PM.

                              Thanks again!

                              Comment


                                Hi, I have just a couple of questions. I'm working with a group for a VR project at our college and we bought the Leap Motion hoping to use it as an in between while we waited for our VR gloves to come in (hopefully some time next month or April) I found your plugin and have been trying to adapt it into our project(which is very basic at the moment) I'm able to get everything to work correctly when I first load the plug in into the project. Play in Editor works great, the leap recognizes both hands all individual finger movements etc.

                                FIRST ISSUE:
                                However when I try and launch standalone, after the whole cooking process I get an error telling me that Leap.DLL could not be found on my computer. I've checked through the files in the project and the files that came with your plugin as well as Leaps SDK. Leap.dll is all over the place, I even copy pasted it to several other locations, still getting this error every time I try and launch stand alone.

                                SECOND ISSUE:
                                After initially loading the Leap plugin into our project and everything working great, I saved everything so as not to lose the setup, as it was working and then closed UE4. When I went to launch the project again later, I get around 430-450 load errors on launch, varying each time I launch the project. I didn't change anything between getting leap to work and closing the project and didn't touch the project between closing it and relaunching it later that same hour.

                                Attached are a couple of screen shots. One showing the content browser from UE4 which shows all of the Leap plugin folders still in the same location I placed them when I got the plugin to work the first time. The other screen shot is just a small number of the Load errors I'm getting when launching after initially getting leap to work.


                                Side note...I can delete the copy of the project I made to test the plugin with (and in the process deleting all of the leap motion items) then create another copy, re-add all of the leap stuff to it and it works again but only in Play in Editor, and only that first time. After I save/close/relaunch it load errors again. And the Standalone has never worked.

                                We are using the latest Leap SDK, with UE 4.6.1
                                Attached Files

                                Comment

                                Working...
                                X