[Plugin] Leap Motion - Event Driven

[a.carter1182]

Thats great vimaxus!! Im trying to think in my head how I can break a mouse apart and use the LMB + guts of mouse and make either a wooden gun or an attachment to our simulated firearms in order to trigger the firing after I've setup the trackers on the item. So for his script did you just have to essentially change things like mathf to fmath, etc for it to work in C++ ?

[vimaxus]

I managed to get the blob tracking working. It's really just the same as Zalo's script so all credit goes to him and of course getnamo. I'll try to send a pm with it to getnamo hopefully to be included in his plugin. (not sure how github branching works and I prefer to avoid it if I can)

https://www.youtube.com/watch?v=QYFN...ature=youtu.be

you can see the red/blue spheres as debug spheres for blob positions in images and green spheres with 3d positions of blobs (the other fixed ones are the halogen lamps)

[a.carter1182]

Ahhh ok, essentially taking it back to capsule collider that is like the default on a character with the camera, etc. I got you thanks for that...didnt think to do that. I figured it would be a long shot on the ps move, and that marker tracking looks useable I really appreciate that link. As always thanks for your great work.

[getnamo]

Maybe Im going about this the wrong way.....but I cant figure out how to make collisions happen with the hands/fingers. The body was already colliding but no setups/checkboxes etc. have seemed to work. Could you provide me with a bit of knowledge please sir.........also quick question, more towards your work with hydra plugin.....but I and most likely others.....havent been able to purchase a hydra system. No telling when STEM is coming out, but I know at some point somebody was using your work I think to attempt the same result with the PS Move. He/She got close but I believe it was some camera drivers or something that stopped it. Anyhow wondering if a PS Move plugin might be a project you might startup one day. Would be nice as I have no solution for shooting in VR that is not immersion breaking yet.....as in I dont want a xbox controller......at least some sort of gun like device the Move does and not really sure if the Hydra has one. Either way great work sir I wish I could get my programming skills up enough that I could contribute to the VR community.

There are a couple ways to get collision to work. Probably the easiest one would be to add box/capsule components which are parented to sockets for fingers/ hand positions. Refer to SkeletalMesh Socket documentation and Collision documentation for details.

I am unlikely to bind the PS Move but if you want to take a crack at it, remember that UE4 requires 64bit dlls in order to use in the editor, which I believe was the issue with the previous attempt by n00854180t. Refer to any of my plugins for reference on how to achieve a plugin bind, but without a 64bit dll you're unlikely to be successful. Best bet is to use devices which are out now like the Leap and Myo or wait for future VR devices. At least one user (Zalo) has been working on tracking other objects through the leap see retroreflective marker forum post for details.


Update to 0.8.2
-Now supports UE 4.6

[a.carter1182]

Maybe Im going about this the wrong way.....but I cant figure out how to make collisions happen with the hands/fingers. The body was already colliding but no setups/checkboxes etc. have seemed to work. Could you provide me with a bit of knowledge please sir.........also quick question, more towards your work with hydra plugin.....but I and most likely others.....havent been able to purchase a hydra system. No telling when STEM is coming out, but I know at some point somebody was using your work I think to attempt the same result with the PS Move. He/She got close but I believe it was some camera drivers or something that stopped it. Anyhow wondering if a PS Move plugin might be a project you might startup one day. Would be nice as I have no solution for shooting in VR that is not immersion breaking yet.....as in I dont want a xbox controller......at least some sort of gun like device the Move does and not really sure if the Hydra has one. Either way great work sir I wish I could get my programming skills up enough that I could contribute to the VR community.

[Xeno]

You'r Fantasticcccccccc
Thank youu *

[getnamo]

One thing off of the top of my head: when you added a leap component to your blueprint, did you use the direct method or event graph method? The reason I ask is that sometimes people make the mistake of doing both, which will duplicate the component, its supposed to be either one or the other, never both. Your log will output a warning any time you have more than one leap component registered (note that while in the editor, it may report a false positive due to the editor adding an extra component during component preview).

If that doesn't solve it, pm me some of your recent logs found in Saved/Logs, any logs which resulted in a crash caused by this would be most helpful in finding the root cause.

With the logs sent from Mega Future Games I was able to identify the memory leaks that were causing this and I pushed an update with these fixes, as well as other API that was missing from LeapFrame.

Update to 0.8.1
-Fixed Memory leaks which would cause crash around 60second mark (GC
interval) for list-based objects using extended and append
-Added appendTools and appendFingers to pointableList
-LeapFrame got its API updated to include all of the missing functions,
should now reflect the leap api nearly completely (except 64bit values
in blueprint that are not supported)
-Added log entry with plugin version at game start for easier debugging

Also added some simple setup gifs to show how easy it is to use the basic rigged setup:
Setting your character


Play Test

[getnamo]

Thanks for getting back to me so quick.

I did as you suggested and deleted the plugin completely and re-added the newer version. Now the engine sees it as ver. 0.8. I reimplemented my code with the updated functions from the new version.

It hasn't seemed to help unfortunately. Ive made a short screen cast showing the problem: https://www.youtube.com/watch?v=g7pu...ature=youtu.be
In the video, the circle gesture event fires repeatedly (each time it changes the colour of the block on the table, has a 1.5 second cool down, then fires again). Once the time hits one minute, it stops firing. Although you cant see my hand, it still preforms the circle gesture after the minute mark.

I assume the same root cause is whats causing the polling gesture to crash as well so I've not yet tested that with the update (that's next on the to do list while I wait a reply). Im sure its something simple I'm just not noticing.

Edit: I tested the Polled Gesture and it did go well either. It goes past the minute marker and still regression the gesture though.The game exits fine with the code as is, if you don't preform the gesture. If you do preform it at least once, the game carries on fine but when you go to exit the game it crashes. It gives different reasons for doing so in the crash report each time.

Still, just letting everyone know. One problem at a time, I guess.

One thing off of the top of my head: when you added a leap component to your blueprint, did you use the direct method or event graph method? The reason I ask is that sometimes people make the mistake of doing both, which will duplicate the component, its supposed to be either one or the other, never both. Your log will output a warning any time you have more than one leap component registered (note that while in the editor, it may report a false positive due to the editor adding an extra component during component preview).

If that doesn't solve it, pm me some of your recent logs found in Saved/Logs, any logs which resulted in a crash caused by this would be most helpful in finding the root cause.

[Mega Future Games]

Thanks for getting back to me so quick.

I did as you suggested and deleted the plugin completely and re-added the newer version. Now the engine sees it as ver. 0.8. I reimplemented my code with the updated functions from the new version.

It hasn't seemed to help unfortunately. Ive made a short screen cast showing the problem: https://www.youtube.com/watch?v=g7pu...ature=youtu.be
In the video, the circle gesture event fires repeatedly (each time it changes the colour of the block on the table, has a 1.5 second cool down, then fires again). Once the time hits one minute, it stops firing. Although you cant see my hand, it still preforms the circle gesture after the minute mark.

I assume the same root cause is whats causing the polling gesture to crash as well so I've not yet tested that with the update (that's next on the to do list while I wait a reply). Im sure its something simple I'm just not noticing.

Edit: I tested the Polled Gesture and it did go well either. It goes past the minute marker and still regression the gesture though.The game exits fine with the code as is, if you don't preform the gesture. If you do preform it at least once, the game carries on fine but when you go to exit the game it crashes. It gives different reasons for doing so in the crash report each time.

Still, just letting everyone know. One problem at a time, I guess.

[getnamo]

Hi getnamo,

Firstly, cheers for all the work you've been doing and all the support you provide. Thanks for my entire team.

I'm having a few issues with the plug-in though. The main one at this time is the plug-in seems to partially cut out after 60 seconds of play.

I currently have 2 custom gestures built into my game, one based off the circle gesture event and one that uses polling to find finger data at a given time. The game starts fine but after 60 seconds, no matter how I preform the circle gesture, the event stops firing. If I try the polling gesture after that time, the engine tends to crash straight away or when I end the game. The game cant seem to find the functions after his point.

I also have some character look code that is based off the players hand position. This works no matter how long the game runs.

I was initially using 0.7.11 and updated to 0.8 today (didn't fix it) but for some reason the engine still says 0.7.11 . From reading this thread, this has been an issue before. Something to do with garbage collection and the plug-in variables, but this had been fixed. Any ideas on what could be causing this?

I spotted a few more incorrect garbage collection links and all of those should be fixed in 0.8. The plugin version definitely says 0.8 so you may not have correctly replaced the plugin. Remember that you need to delete the old plugin content/plugin source to replace it as some classes have been removed.

If you're still getting an issue, try to narrow the function call as much as you can. I believe yours seems to be due to the controller->frame releasing the object early which should have been fixed in the last update.

I've been using the 0.8 version in the leapjam and it's been stable while using the circle gesture detection for quite some time, way beyond the GC interval. Let me know how it goes!

Awesome getnamo,
Thanks again for this. I'll have this up and running for testing soon. Looking forward to it

Great, looking forward to the feedback

[Mega Future Games]

Hi getnamo,

Firstly, cheers for all the work you've been doing and all the support you provide. Thanks for my entire team.

I'm having a few issues with the plug-in though. The main one at this time is the plug-in seems to partially cut out after 60 seconds of play.

I currently have 2 custom gestures built into my game, one based off the circle gesture event and one that uses polling to find finger data at a given time. The game starts fine but after 60 seconds, no matter how I preform the circle gesture, the event stops firing. If I try the polling gesture after that time, the engine tends to crash straight away or when I end the game. The game cant seem to find the functions after his point.

I also have some character look code that is based off the players hand position. This works no matter how long the game runs.

I was initially using 0.7.11 and updated to 0.8 today (didn't fix it) but for some reason the engine still says 0.7.11 . From reading this thread, this has been an issue before. Something to do with garbage collection and the plug-in variables, but this had been fixed. Any ideas on what could be causing this?

[a.carter1182]

Awesome getnamo,
Thanks again for this. I'll have this up and running for testing soon. Looking forward to it

[getnamo]

I completely understand,
I have a cyber security visualization (which is what I'd like to use the ironman ui for) and another project for virtual training, and well.......to say the least I understand multi-tasking projects . Anyhow thanks again for all your hard work and I look forward to working with your plugin.

Pushed an update to 0.8
-Basic Rigged leap character added. Works like the LeapDebugCharacter,
change your player controller to VRPlayercontroller and pawn to
LeapRiggedCharacter
-Updated bind to Leap SDK 2.2.0
-Ensured consistency by prepending leap to all classes, this will likely
break compatibility with old blueprints.
-Fixed dynamic object allocation architecture
-Added basic AnimBody, Hand, Finger, and Bone for convenient animation
state and basis conversion.


To use the rigged character set your PlayerController to VRPlayerController and your pawn to LeapRiggedCharacter. Please note that this is a preliminary support for rigging and there will likely be bugs. Also it is important to note that the namespace update for the classes will likely break compatibility with your old blueprint setups.

Will provide more documentation sometime after the jam.

First off, thank you for all of your work, the plugin is absolutely fantastic.

I just have one question - Every time I use a "Set Timer" node, various errors similar to this are thrown
...
It directs me to nodes in the functions I set up in my timers - and if I leave the functions empty, directly to the function call. Have you encountered this or know why this would happen? Thank you!

Seems like some of the nodes are returning null objects (none), identify the culprit and do a cast or != none check to suppress the warning.

[Tarwine]

First off, thank you for all of your work, the plugin is absolutely fantastic.

I just have one question - Every time I use a "Set Timer" node, various errors similar to this are thrown

LogScript:Warning: Attempt to assign variable through None
CarriageCharacter_C /Game/Maps/UEDPIE_0_Level_ver1.Level_ver1:PersistentLevel.CarriageCharacter_C_29
Function /Game/Blueprints/CarriageCharacter.CarriageCharacter_C:ExecuteUbergraph_CarriageCharacter:110E
PIE:Error: Error Attempt to assign variable through None from node Set ReinsGestureDetected in blueprint CarriageCharacter

LogScript:Warning: Accessed None 'K2Node_Event_hand'
CarriageCharacter_C /Game/Maps/UEDPIE_0_Level_ver1.Level_ver1:PersistentLevel.CarriageCharacter_C_0
Function /Game/Blueprints/CarriageCharacter.CarriageCharacter_C:ExecuteUbergraph_CarriageCharacter:0941
PIE:Error: Error Accessed None 'K2Node_Event_hand' from node Play in blueprint CarriageCharacter

It directs me to nodes in the functions I set up in my timers - and if I leave the functions empty, directly to the function call. Have you encountered this or know why this would happen? Thank you!

[a.carter1182]

I completely understand,
I have a cyber security visualization (which is what I'd like to use the ironman ui for) and another project for virtual training, and well.......to say the least I understand multi-tasking projects . Anyhow thanks again for all your hard work and I look forward to working with your plugin.