[Plugin] Leap Motion - Event Driven

[getnamo]

I managed to get the blob tracking working. It's really just the same as Zalo's script so all credit goes to him and of course getnamo. I'll try to send a pm with it to getnamo hopefully to be included in his plugin. (not sure how github branching works and I prefer to avoid it if I can)

https://www.youtube.com/watch?v=QYFN...ature=youtu.be

you can see the red/blue spheres as debug spheres for blob positions in images and green spheres with 3d positions of blobs (the other fixed ones are the halogen lamps)

This is pretty cool! I noticed you made a pull request, I will take a look and see how to best integrate this. What is it that you are tracking in this example?

This plugin is not supported for 4.6th version

And

In 4.5th version Plugin is supporting well, but while exporting to .EXE for windows 64 and 32 bit it rises error, its not working....

I'm using windows 8 64 bit

...

I may need more information, describe in more detail what you are trying to do. When you say exporting to .exe, do you mean packaging for shipping/development? Please make sure you follow the shipping instructions found at the github repo if that is the case. If you are still having problems, PM me your logs found in {Project Root}/Saved/Logs

[a.carter1182]

Works fine on my 4.6 install with getnamo's latest + leap 2.2.......I've never tried to export yet so I cant comment there.

[Rasa]

This plugin is not supported for 4.6th version

And

In 4.5th version Plugin is supporting well, but while exporting to .EXE for windows 64 and 32 bit it rises error, its not working....

I'm using windows 8 64 bit

[a.carter1182]

Heh......thought that was a sarcastic remark at first........but that seems to be a legit software/hardware that I really might be able to make work for this application. Great job on the code work and I'll be able to push further as this comes along. Also thanks getnamo as the collision now works after sockets with some cubes attached.

[vimaxus]

Thats great vimaxus!! Im trying to think in my head how I can break a mouse apart and use the LMB + guts of mouse and make either a wooden gun or an attachment to our simulated firearms in order to trigger the firing after I've setup the trackers on the item. So for his script did you just have to essentially change things like mathf to fmath, etc for it to work in C++ ?

http://www.makeymakey.com/ ?

it's not that straightforward, you'll have access to the source soon enough. I'm also working on a blueprint to lock on a certain blob (or you could just sort the blobs) because for example the 3rd one from the vector array isn't necessarily the same blob all the time.

[a.carter1182]

Thats great vimaxus!! Im trying to think in my head how I can break a mouse apart and use the LMB + guts of mouse and make either a wooden gun or an attachment to our simulated firearms in order to trigger the firing after I've setup the trackers on the item. So for his script did you just have to essentially change things like mathf to fmath, etc for it to work in C++ ?

[vimaxus]

I managed to get the blob tracking working. It's really just the same as Zalo's script so all credit goes to him and of course getnamo. I'll try to send a pm with it to getnamo hopefully to be included in his plugin. (not sure how github branching works and I prefer to avoid it if I can)

https://www.youtube.com/watch?v=QYFN...ature=youtu.be

you can see the red/blue spheres as debug spheres for blob positions in images and green spheres with 3d positions of blobs (the other fixed ones are the halogen lamps)

[a.carter1182]

Ahhh ok, essentially taking it back to capsule collider that is like the default on a character with the camera, etc. I got you thanks for that...didnt think to do that. I figured it would be a long shot on the ps move, and that marker tracking looks useable I really appreciate that link. As always thanks for your great work.

[getnamo]

Maybe Im going about this the wrong way.....but I cant figure out how to make collisions happen with the hands/fingers. The body was already colliding but no setups/checkboxes etc. have seemed to work. Could you provide me with a bit of knowledge please sir.........also quick question, more towards your work with hydra plugin.....but I and most likely others.....havent been able to purchase a hydra system. No telling when STEM is coming out, but I know at some point somebody was using your work I think to attempt the same result with the PS Move. He/She got close but I believe it was some camera drivers or something that stopped it. Anyhow wondering if a PS Move plugin might be a project you might startup one day. Would be nice as I have no solution for shooting in VR that is not immersion breaking yet.....as in I dont want a xbox controller......at least some sort of gun like device the Move does and not really sure if the Hydra has one. Either way great work sir I wish I could get my programming skills up enough that I could contribute to the VR community.

There are a couple ways to get collision to work. Probably the easiest one would be to add box/capsule components which are parented to sockets for fingers/ hand positions. Refer to SkeletalMesh Socket documentation and Collision documentation for details.

I am unlikely to bind the PS Move but if you want to take a crack at it, remember that UE4 requires 64bit dlls in order to use in the editor, which I believe was the issue with the previous attempt by n00854180t. Refer to any of my plugins for reference on how to achieve a plugin bind, but without a 64bit dll you're unlikely to be successful. Best bet is to use devices which are out now like the Leap and Myo or wait for future VR devices. At least one user (Zalo) has been working on tracking other objects through the leap see retroreflective marker forum post for details.


Update to 0.8.2
-Now supports UE 4.6

[a.carter1182]

Maybe Im going about this the wrong way.....but I cant figure out how to make collisions happen with the hands/fingers. The body was already colliding but no setups/checkboxes etc. have seemed to work. Could you provide me with a bit of knowledge please sir.........also quick question, more towards your work with hydra plugin.....but I and most likely others.....havent been able to purchase a hydra system. No telling when STEM is coming out, but I know at some point somebody was using your work I think to attempt the same result with the PS Move. He/She got close but I believe it was some camera drivers or something that stopped it. Anyhow wondering if a PS Move plugin might be a project you might startup one day. Would be nice as I have no solution for shooting in VR that is not immersion breaking yet.....as in I dont want a xbox controller......at least some sort of gun like device the Move does and not really sure if the Hydra has one. Either way great work sir I wish I could get my programming skills up enough that I could contribute to the VR community.

[Xeno]

You'r Fantasticcccccccc
Thank youu *

[getnamo]

One thing off of the top of my head: when you added a leap component to your blueprint, did you use the direct method or event graph method? The reason I ask is that sometimes people make the mistake of doing both, which will duplicate the component, its supposed to be either one or the other, never both. Your log will output a warning any time you have more than one leap component registered (note that while in the editor, it may report a false positive due to the editor adding an extra component during component preview).

If that doesn't solve it, pm me some of your recent logs found in Saved/Logs, any logs which resulted in a crash caused by this would be most helpful in finding the root cause.

With the logs sent from Mega Future Games I was able to identify the memory leaks that were causing this and I pushed an update with these fixes, as well as other API that was missing from LeapFrame.

Update to 0.8.1
-Fixed Memory leaks which would cause crash around 60second mark (GC
interval) for list-based objects using extended and append
-Added appendTools and appendFingers to pointableList
-LeapFrame got its API updated to include all of the missing functions,
should now reflect the leap api nearly completely (except 64bit values
in blueprint that are not supported)
-Added log entry with plugin version at game start for easier debugging

Also added some simple setup gifs to show how easy it is to use the basic rigged setup:
Setting your character


Play Test

[getnamo]

Thanks for getting back to me so quick.

I did as you suggested and deleted the plugin completely and re-added the newer version. Now the engine sees it as ver. 0.8. I reimplemented my code with the updated functions from the new version.

It hasn't seemed to help unfortunately. Ive made a short screen cast showing the problem: https://www.youtube.com/watch?v=g7pu...ature=youtu.be
In the video, the circle gesture event fires repeatedly (each time it changes the colour of the block on the table, has a 1.5 second cool down, then fires again). Once the time hits one minute, it stops firing. Although you cant see my hand, it still preforms the circle gesture after the minute mark.

I assume the same root cause is whats causing the polling gesture to crash as well so I've not yet tested that with the update (that's next on the to do list while I wait a reply). Im sure its something simple I'm just not noticing.

Edit: I tested the Polled Gesture and it did go well either. It goes past the minute marker and still regression the gesture though.The game exits fine with the code as is, if you don't preform the gesture. If you do preform it at least once, the game carries on fine but when you go to exit the game it crashes. It gives different reasons for doing so in the crash report each time.

Still, just letting everyone know. One problem at a time, I guess.

One thing off of the top of my head: when you added a leap component to your blueprint, did you use the direct method or event graph method? The reason I ask is that sometimes people make the mistake of doing both, which will duplicate the component, its supposed to be either one or the other, never both. Your log will output a warning any time you have more than one leap component registered (note that while in the editor, it may report a false positive due to the editor adding an extra component during component preview).

If that doesn't solve it, pm me some of your recent logs found in Saved/Logs, any logs which resulted in a crash caused by this would be most helpful in finding the root cause.

[Mega Future Games]

Thanks for getting back to me so quick.

I did as you suggested and deleted the plugin completely and re-added the newer version. Now the engine sees it as ver. 0.8. I reimplemented my code with the updated functions from the new version.

It hasn't seemed to help unfortunately. Ive made a short screen cast showing the problem: https://www.youtube.com/watch?v=g7pu...ature=youtu.be
In the video, the circle gesture event fires repeatedly (each time it changes the colour of the block on the table, has a 1.5 second cool down, then fires again). Once the time hits one minute, it stops firing. Although you cant see my hand, it still preforms the circle gesture after the minute mark.

I assume the same root cause is whats causing the polling gesture to crash as well so I've not yet tested that with the update (that's next on the to do list while I wait a reply). Im sure its something simple I'm just not noticing.

Edit: I tested the Polled Gesture and it did go well either. It goes past the minute marker and still regression the gesture though.The game exits fine with the code as is, if you don't preform the gesture. If you do preform it at least once, the game carries on fine but when you go to exit the game it crashes. It gives different reasons for doing so in the crash report each time.

Still, just letting everyone know. One problem at a time, I guess.

[getnamo]

Hi getnamo,

Firstly, cheers for all the work you've been doing and all the support you provide. Thanks for my entire team.

I'm having a few issues with the plug-in though. The main one at this time is the plug-in seems to partially cut out after 60 seconds of play.

I currently have 2 custom gestures built into my game, one based off the circle gesture event and one that uses polling to find finger data at a given time. The game starts fine but after 60 seconds, no matter how I preform the circle gesture, the event stops firing. If I try the polling gesture after that time, the engine tends to crash straight away or when I end the game. The game cant seem to find the functions after his point.

I also have some character look code that is based off the players hand position. This works no matter how long the game runs.

I was initially using 0.7.11 and updated to 0.8 today (didn't fix it) but for some reason the engine still says 0.7.11 . From reading this thread, this has been an issue before. Something to do with garbage collection and the plug-in variables, but this had been fixed. Any ideas on what could be causing this?

I spotted a few more incorrect garbage collection links and all of those should be fixed in 0.8. The plugin version definitely says 0.8 so you may not have correctly replaced the plugin. Remember that you need to delete the old plugin content/plugin source to replace it as some classes have been removed.

If you're still getting an issue, try to narrow the function call as much as you can. I believe yours seems to be due to the controller->frame releasing the object early which should have been fixed in the last update.

I've been using the 0.8 version in the leapjam and it's been stable while using the circle gesture detection for quite some time, way beyond the GC interval. Let me know how it goes!

Awesome getnamo,
Thanks again for this. I'll have this up and running for testing soon. Looking forward to it

Great, looking forward to the feedback