Announcement

Collapse
No announcement yet.

[Plugin] Leap Motion - Event Driven

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    Originally posted by a.carter1182 View Post
    Maybe Im going about this the wrong way.....but I cant figure out how to make collisions happen with the hands/fingers. The body was already colliding but no setups/checkboxes etc. have seemed to work. Could you provide me with a bit of knowledge please sir.........also quick question, more towards your work with hydra plugin.....but I and most likely others.....havent been able to purchase a hydra system. No telling when STEM is coming out, but I know at some point somebody was using your work I think to attempt the same result with the PS Move. He/She got close but I believe it was some camera drivers or something that stopped it. Anyhow wondering if a PS Move plugin might be a project you might startup one day. Would be nice as I have no solution for shooting in VR that is not immersion breaking yet.....as in I dont want a xbox controller......at least some sort of gun like device the Move does and not really sure if the Hydra has one. Either way great work sir I wish I could get my programming skills up enough that I could contribute to the VR community.
    There are a couple ways to get collision to work. Probably the easiest one would be to add box/capsule components which are parented to sockets for fingers/ hand positions. Refer to SkeletalMesh Socket documentation and Collision documentation for details.

    I am unlikely to bind the PS Move but if you want to take a crack at it, remember that UE4 requires 64bit dlls in order to use in the editor, which I believe was the issue with the previous attempt by n00854180t. Refer to any of my plugins for reference on how to achieve a plugin bind, but without a 64bit dll you're unlikely to be successful. Best bet is to use devices which are out now like the Leap and Myo or wait for future VR devices. At least one user (Zalo) has been working on tracking other objects through the leap see retroreflective marker forum post for details.


    Update to 0.8.2
    -Now supports UE 4.6
    Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

    Comment


      Ahhh ok, essentially taking it back to capsule collider that is like the default on a character with the camera, etc. I got you thanks for that...didnt think to do that. I figured it would be a long shot on the ps move, and that marker tracking looks useable I really appreciate that link. As always thanks for your great work.
      Twitter: https://twitter.com/a_carter1182

      Comment


        I managed to get the blob tracking working. It's really just the same as Zalo's script so all credit goes to him and of course getnamo. I'll try to send a pm with it to getnamo hopefully to be included in his plugin. (not sure how github branching works and I prefer to avoid it if I can)

        https://www.youtube.com/watch?v=QYFN...ature=youtu.be

        you can see the red/blue spheres as debug spheres for blob positions in images and green spheres with 3d positions of blobs (the other fixed ones are the halogen lamps)
        Last edited by vimaxus; 12-10-2014, 10:14 AM.

        Comment


          Thats great vimaxus!! Im trying to think in my head how I can break a mouse apart and use the LMB + guts of mouse and make either a wooden gun or an attachment to our simulated firearms in order to trigger the firing after I've setup the trackers on the item. So for his script did you just have to essentially change things like mathf to fmath, etc for it to work in C++ ?
          Twitter: https://twitter.com/a_carter1182

          Comment


            Originally posted by a.carter1182 View Post
            Thats great vimaxus!! Im trying to think in my head how I can break a mouse apart and use the LMB + guts of mouse and make either a wooden gun or an attachment to our simulated firearms in order to trigger the firing after I've setup the trackers on the item. So for his script did you just have to essentially change things like mathf to fmath, etc for it to work in C++ ?
            http://www.makeymakey.com/ ?

            it's not that straightforward, you'll have access to the source soon enough. I'm also working on a blueprint to lock on a certain blob (or you could just sort the blobs) because for example the 3rd one from the vector array isn't necessarily the same blob all the time.
            Last edited by vimaxus; 12-10-2014, 11:57 AM.

            Comment


              Heh......thought that was a sarcastic remark at first........but that seems to be a legit software/hardware that I really might be able to make work for this application. Great job on the code work and I'll be able to push further as this comes along. Also thanks getnamo as the collision now works after sockets with some cubes attached.
              Twitter: https://twitter.com/a_carter1182

              Comment


                This plugin is not supported for 4.6th version

                And

                In 4.5th version Plugin is supporting well, but while exporting to .EXE for windows 64 and 32 bit it rises error, its not working....

                I'm using windows 8 64 bit

                Click image for larger version

Name:	Ur.jpg
Views:	1
Size:	50.5 KB
ID:	1062026

                Comment


                  Works fine on my 4.6 install with getnamo's latest + leap 2.2.......I've never tried to export yet so I cant comment there.
                  Twitter: https://twitter.com/a_carter1182

                  Comment


                    Originally posted by vimaxus
                    I managed to get the blob tracking working. It's really just the same as Zalo's script so all credit goes to him and of course getnamo. I'll try to send a pm with it to getnamo hopefully to be included in his plugin. (not sure how github branching works and I prefer to avoid it if I can)

                    https://www.youtube.com/watch?v=QYFN...ature=youtu.be

                    you can see the red/blue spheres as debug spheres for blob positions in images and green spheres with 3d positions of blobs (the other fixed ones are the halogen lamps)
                    This is pretty cool! I noticed you made a pull request, I will take a look and see how to best integrate this. What is it that you are tracking in this example?


                    Originally posted by Rasa View Post
                    This plugin is not supported for 4.6th version

                    And

                    In 4.5th version Plugin is supporting well, but while exporting to .EXE for windows 64 and 32 bit it rises error, its not working....

                    I'm using windows 8 64 bit

                    ...
                    I may need more information, describe in more detail what you are trying to do. When you say exporting to .exe, do you mean packaging for shipping/development? Please make sure you follow the shipping instructions found at the github repo if that is the case. If you are still having problems, PM me your logs found in {Project Root}/Saved/Logs
                    Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                    Comment


                      Originally posted by getnamo View Post
                      This is pretty cool! I noticed you made a pull request, I will take a look and see how to best integrate this. What is it that you are tracking in this example?
                      Well, I was tracking just some blobs in the image. If you mean in real-life, as I couldn't find what zalo suggested (motion tracking spheres) I covered some polystyrene balls (3.5cm) with 3M reflecting tape to get sort of equivalent.
                      I tried with small pathes initially (1.5cm squares) on my mobile phone but if I went over 30-40cm away from the leap I got very inconsistent results. The resolution is very low on the leap, well, actually the resolution coupled with the huge angle gives a too greater solid angle for each pixel so you need BIG things to track.

                      Here is how I used it at the time:
                      Click image for larger version

Name:	track_bluep1.jpg
Views:	1
Size:	159.3 KB
ID:	1062100

                      As I said in the note, I find the CPU use exaggerated, so I'm trying to export 3 vectors for each blob instead of one, the other 2 being coordinates on the image, in order to be able to use some condition so that when I satisfy a list of conditions of relative distances from each other. (like a triange, or a sqare with diagonals) I "lock" only on the blobs and only search in the vicinity of where they were the last frame. It's looking great, it's a lot better CPU wise, but now I'm trying to change the recursive flood with just a regular pass on a square surrounding the last position.

                      Also, if anyone knows some algorithms to eliminate a high-frequency shake from my measurements (but not averaging as not to introduce latency) I'm very interested. I remember reading something of the sort on an Oculus blob post, quite a while back.

                      edit: I initially worked on a version in LeapImageList so as to do directly the 2 images and return directly the 3d blobs, to avoid the 2 separate branches in blueprint, but while it worked I got often crashes (sometimes around the 60sec mark, sometimes later) relating to setLeapImage. Maybe I was doing something wrong but I basically kept the 2 pointers that you used and re-defined them at each call. If you want I can send the related files, but I eventually thought that maybe, someone would like the separate results by image...
                      Last edited by vimaxus; 12-14-2014, 09:19 AM.

                      Comment


                        i have a problem with rotation of the hands of my custom character. its rigged with ART.

                        Anyone known where are the problem? in the video i try clapping and show my palm in front of my head.

                        Here the video:
                        https://www.youtube.com/watch?v=1VXQdHgom6g

                        The code its copy the necessary nodes from plugin content, but not all, perhaps i missed something?
                        Last edited by knack; 12-15-2014, 07:24 AM.
                        pd: excuse my english.

                        lfw/paid modeling, painting, texturing.

                        Comment


                          ... duplicate ...
                          pd: excuse my english.

                          lfw/paid modeling, painting, texturing.

                          Comment


                            Originally posted by knack View Post
                            i have a problem with rotation of the hands of my custom character. its rigged with ART.

                            Anyone known where are the problem? in the video i try clapping and show my palm in front of my head.

                            Here the video:
                            https://www.youtube.com/watch?v=1VXQdHgom6g

                            The code its copy the necessary nodes from plugin content, but not all, perhaps i missed something?
                            It looks like your hands are rotating around YAW whenever they cross some midpoint line. I encountered something similar with the fingers in the rigged model and didn't have the time to correct it, but the wrist rotation doesn't have this problem. It important to remember that you need to combine rotations when you change orientation in blueprint and to avoid using break rotation/make rotation unless you know what you're doing. If you do not heed that, you may get gimbal locks.

                            The general format of the rigging looks like this: Leap Positions and Orientations in UE space (X forward), Leap basis (Leap Debug Hand location) subtracted from component (converts leap position to component space) -> Rotated by 90 degrees to have Y forward for animation space. Use these rotations/positions in your rigging.
                            Plugins: Node.js - TensorFlow - Socket.io Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense

                            Comment


                              WTF, after hours and hours, and near do a harakiri ritual... i found what was failing.

                              My anigraph was something like this...: (i not remember exact)

                              LeftElbowTransform->RightHandTransform->RightElbowTransform->LeftHandTransform


                              i change to the order to your template order and fix the issue... dammit!



                              thanks for the plugin work. Still looking step by step how works i not understand few thinks of that blueprints. Ins't basic your example! :lol:
                              pd: excuse my english.

                              lfw/paid modeling, painting, texturing.

                              Comment


                                I supposed that its the finger problem:
                                https://www.youtube.com/watch?v=Ya_s3WLLAJI
                                I found too a problem when move the HMD (last minute in video i think) the arms move. Your example proyect have same problem.
                                pd: excuse my english.

                                lfw/paid modeling, painting, texturing.

                                Comment

                                Working...
                                X