No announcement yet.

Gesture Tracker VR Help and Feature Requests

  • Filter
  • Time
  • Show
Clear All
new posts

    Originally posted by hdelattre View Post

    Hi @quickflint,

    I decided to leave that kind of functionality up to manual setup by users. You can just set up a gesture tracker on each hand, record your circle gesture with id 0 for each of them, and then check if they happen in the right order and time frame. So I when the left hand circle is recognized I would record a timestamp using GetWorldTimeSeconds, and whenever the right hand push is recognized I would check if (GetWorldTimeSeconds - CircleGestureTimestamp) < TimeWindow, where TimeWindow is something like 1 or 2 seconds that you have between drawing the cirlce and pushing through it.
    Thank you! That is awesome!
    I will be grabbing the plugin. Really happy to see you are still active in the support forum. Thank you very much for your response.


      Hi, I'm trying to save and use gestures for both controllers simultaneously (UE 4.19.2). I'm hitting a few different snags on this.

      1. I can't successfully save both sets. Some gestures will show up in one file but not the other. Some don't save at all. Here's my record/save/load set up:
      Click image for larger version

Name:	image_140557.jpg
Views:	40
Size:	198.4 KB
ID:	1491414 I've also tried using one each of the Start/Finish Recording nodes with both trackers plugged in as the target. I get the same result either way.

      2. I don't know how to get the below nodes created. I assume I need one to recognize the Left controller but I can't find how to create these nodes. They do not show up in the right-click menu.
      Click image for larger version

Name:	image_140558.jpg
Views:	41
Size:	33.5 KB
ID:	1491415



        Hi ForwardXPDev ,

        1. Each tracker saves its own set of gestures so you shouldn't expect any from one file to show up in the other. Based on your setup my best guess for why some gestures are disappearing is that maybe you're recording to the same Id multiple times, which overwrites the gesture previously recorded to that Id. Also, gesture recording fails when the recorded gesture's length is greater than the maximum allowed length (you can tweak this in the tracker's details panel), so I suppose it's possible you're recording very long gestures sometimes and they're failing. Your setup looks right (although the delay between saves is probably unnecessary) so I'm not sure what the issue would be otherwise.

        2. If you click on one of the GestureTracker components and look at the bottom of the details panel you'll see some events with '+' symbols next to them. Click these to create the events shown above. Alternatively you can drag a pin off a tracker and call BindToGestureRecognized, then drag off the resulting bind node's delegate pin and create a custom event (this will auto generate an event with the proper signature, or inputs).
        Last edited by hdelattre; 06-20-2018, 12:13 AM.
        Gesture Tracker VR: A Gesture Recognition Plugin


          Can u get the speed of how fast the tracker completes the pattern? or if it only completes 50%?


            Hi calvin019,

            The Recognition Duration output on the gesture recognized event tells you how long it took (see ForwardXPDev's images above).

            If you want to recognize gestures at 50% you can lower the Completion Percentage parameter on the GestureTracker to .5. I may add support for setting individualized completion updates in the future (e.g. have the tracker tell you when a specific gesture id has reached a specific percentage such as 50%). As an alternative for now you could record a gesture that's the 50% version of the full gesture and then check for that from the GestureRecognized event, which will fire for both the 50% gesture and the 100% gesture if both are performed during recognition (instead of just using the result from FinishRecognition, which only cares about the final gesture you performed and doesn't fire until user input tells it that the gesture is done being performed).
            Last edited by hdelattre; 08-11-2018, 04:54 PM.
            Gesture Tracker VR: A Gesture Recognition Plugin



              This plugin is really well done ! However I am having trouble with the gesture library management. There is no way to remove a gesture from the library using blueprints which forces me to redo my gestures very often. Loading multiple files into the same library would also do the job since I could save separately every gestures and just load back together the gestures I want to keep.

              My goal is simply to have a way to keep the gestures I did well and remove the ones I don't want to use anymore. I'm talking about development gestures. In the game I will dynamically use the IDs to enable or disable gestures depending on the context. Basically, I want to add, remove and modify the gestures stored in my tracker so it is faster to create the final library I will package with my game. Using IDs during development doesn't make sense to me since I'll accumulate a few hundred useless gestures in my gesture file at the rate I'm trying things out. I'm fine with writing C++ code but I don't want to modify your plugin and there is no virtual functions I can inherit to override the saving and loading behavior in GestureLibrary.

              Best regards !


                Hi wilbros,

                You're right that gesture management could be much better, and I'm hoping to improve that (among other things) when I find the time. I'll let you know when that happens. In the meantime, I recommend using the following process to record your gesture set:

                1. Bind the gesture save and load functions to some input temporarily.
                2. Start recording your gestures. If you're satisfied with the last one you recorded click the save button. If not, click load to clear the last gesture and repeat until you're satisfied. I also recommend testing performing the gesture before saving it to ensure it feels right.
                Last edited by hdelattre; 10-16-2018, 08:44 PM.
                Gesture Tracker VR: A Gesture Recognition Plugin



                  I saw this was mentioned previously in the thread, but how can I normalize the gestures to be used in any direction? (punching up in the air was the example)

                  Also is it possible to "flatten" gestures as some gestures are causing slight unwanted rotations on the actors I'm trying to spawn.
                  Last edited by 22vortex22; 12-23-2018, 05:02 AM.



                    Theres a tracking parameter since 4.17 or so for normalizing yaw rotation, just turn that on.

                    As for flattening, there’s no way of doing that currently but you could just flatten the rotation you’re spawning the actor with manually.
                    Gesture Tracker VR: A Gesture Recognition Plugin


                      Hi there,

                      This plugin is really well done! I got it for a VR FPP project in which the player will be controlling a bird. I managed to adapt the code to fly but my movement is not as smooth as the one you set up in your You Tube demo. I was wondering if you could share your code by chance? Also is there a way to allow the engine for continuous recognition of the recorded gesture without constantly pressing the trigger and performing one action at the time? Thanks in advance



                        In the second part of the setup video I show how to set up continuous recognition. It's mostly the same, you just set up a different event and call a different function to start it.

                        The flight movement in the video was pretty basic. I had a velocity vector that I used to move the pawn each tick (AddActorOffset w/ velocity * DetaTime). I subtracted 981 units per sec on the z axis for gravity, and each time they trigger the flap gesture on either hand it would added a fairly large amount (try 3000 and tweak it) to the z axis and a small amount in the direction the player is facing to make them start moving forward.

                        To be honest though, if I were making a game with flying as a core mechanic I wouldn't recommend using gestures. You'll get much more nuanced feedback if you specifically program flight mechanics that react precisely to how the player is flapping their arms. I did this for a game of mine and the result is much better than the flight you see at the end of the gesture video. I recommend gestures for scenarios more like the fireball shooting, where you want to give players unique ways to cast spells or send commands. Any psychics mechanics like flight should employ models of that physics interaction instead of depending on fixed gestures. With that said, if you're just trying to prototype the flying the flapping gestures are still plenty of fun, and it only took me about 20 minutes to set that up (versus my better flight implementation that took many hours of testing and refinement).
                        Gesture Tracker VR: A Gesture Recognition Plugin


                          I have an interesting issue, it seems that after I record my gestures, if a taller user plays with the same gestures, it completes too early. How should I tackle this? if it's any help I am trying to create a punch gesture (my arms only reach so far).


                            Hi wandbrandon ,

                            Sorry for the late response. Scale down the trackRecognitionRatio and/or continuousRecognitionRatio parameters for taller players. By default they're set to high multipliers since shape is usually more important than distance for determining what gesture is being performed, but if you're straight line punch gestures you'll probably want to change them to 1 for the tallest player and scale up to around 1.5 for shorter players.
                            Gesture Tracker VR: A Gesture Recognition Plugin


                              Thank you hdelattre !

                              Appreciate the response, unfortunately the plugin is not updated for 4.24. Do you have any intention on updating?


                                wandbrandon sorry for the delay I'll have it up soon. In the meantime if you create a Plugins folder in your project folder and copy the 4.23 GestureTrackerVR plugin folder into there, change the uplugin file "EngineAssociation" to 4.24, and compile it should build fine as is for 4.24.
                                Last edited by hdelattre; 12-30-2019, 05:26 PM.
                                Gesture Tracker VR: A Gesture Recognition Plugin