This plugin is really well done ! However I am having trouble with the gesture library management. There is no way to remove a gesture from the library using blueprints which forces me to redo my gestures very often. Loading multiple files into the same library would also do the job since I could save separately every gestures and just load back together the gestures I want to keep.
My goal is simply to have a way to keep the gestures I did well and remove the ones I don’t want to use anymore. I’m talking about development gestures. In the game I will dynamically use the IDs to enable or disable gestures depending on the context. Basically, I want to add, remove and modify the gestures stored in my tracker so it is faster to create the final library I will package with my game. Using IDs during development doesn’t make sense to me since I’ll accumulate a few hundred useless gestures in my gesture file at the rate I’m trying things out. I’m fine with writing C++ code but I don’t want to modify your plugin and there is no virtual functions I can inherit to override the saving and loading behavior in GestureLibrary.
You’re right that gesture management could be much better, and I’m hoping to improve that (among other things) when I find the time. I’ll let you know when that happens. In the meantime, I recommend using the following process to record your gesture set:
Bind the gesture save and load functions to some input temporarily.
Start recording your gestures. If you’re satisfied with the last one you recorded click the save button. If not, click load to clear the last gesture and repeat until you’re satisfied. I also recommend testing performing the gesture before saving it to ensure it feels right.
I saw this was mentioned previously in the thread, but how can I normalize the gestures to be used in any direction? (punching up in the air was the example)
Also is it possible to “flatten” gestures as some gestures are causing slight unwanted rotations on the actors I’m trying to spawn.
This plugin is really well done! I got it for a VR FPP project in which the player will be controlling a bird. I managed to adapt the code to fly but my movement is not as smooth as the one you set up in your You Tube demo. I was wondering if you could share your code by chance? Also is there a way to allow the engine for continuous recognition of the recorded gesture without constantly pressing the trigger and performing one action at the time? Thanks in advance
In the second part of the setup video I show how to set up continuous recognition. It’s mostly the same, you just set up a different event and call a different function to start it.
The flight movement in the video was pretty basic. I had a velocity vector that I used to move the pawn each tick (AddActorOffset w/ velocity * DetaTime). I subtracted 981 units per sec on the z axis for gravity, and each time they trigger the flap gesture on either hand it would added a fairly large amount (try 3000 and tweak it) to the z axis and a small amount in the direction the player is facing to make them start moving forward.
To be honest though, if I were making a game with flying as a core mechanic I wouldn’t recommend using gestures. You’ll get much more nuanced feedback if you specifically program flight mechanics that react precisely to how the player is flapping their arms. I did this for a game of mine and the result is much better than the flight you see at the end of the gesture video. I recommend gestures for scenarios more like the fireball shooting, where you want to give players unique ways to cast spells or send commands. Any psychics mechanics like flight should employ models of that physics interaction instead of depending on fixed gestures. With that said, if you’re just trying to prototype the flying the flapping gestures are still plenty of fun, and it only took me about 20 minutes to set that up (versus my better flight implementation that took many hours of testing and refinement).
I have an interesting issue, it seems that after I record my gestures, if a taller user plays with the same gestures, it completes too early. How should I tackle this? if it’s any help I am trying to create a punch gesture (my arms only reach so far).
Sorry for the late response. Scale down the trackRecognitionRatio and/or continuousRecognitionRatio parameters for taller players. By default they’re set to high multipliers since shape is usually more important than distance for determining what gesture is being performed, but if you’re straight line punch gestures you’ll probably want to change them to 1 for the tallest player and scale up to around 1.5 for shorter players.
@wandbrandon sorry for the delay I’ll have it up soon. In the meantime if you create a Plugins folder in your project folder and copy the 4.23 GestureTrackerVR plugin folder into there, change the uplugin file “EngineAssociation” to 4.24, and compile it should build fine as is for 4.24.
EDIT: I tried to do it but there are a couple build errors when compiling
Here is the output
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/GestureTracker.cpp(14): warning C4996: ‘USceneComponent::bVisible’: This member will be made private. Please use IsVisible or SetVisibility. Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.
1> C:\Program Files\Epic Games\UE_4.24\Engine\Source\Runtime\Engine\Classes\Components/SceneComponent.h(204): note: see declaration of ‘USceneComponent::bVisible’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(30): error C4458: declaration of ‘startLocation’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(45): note: see declaration of ‘Gesture::startLocation’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(30): error C4458: declaration of ‘rotator’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(48): note: see declaration of ‘Gesture::rotator’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(30): error C4458: declaration of ‘resolution’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(42): note: see declaration of ‘Gesture::resolution’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(35): error C4458: declaration of ‘startLocation’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(45): note: see declaration of ‘Gesture::startLocation’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(35): error C4458: declaration of ‘rotator’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(48): note: see declaration of ‘Gesture::rotator’
1>C:/Users/Brandon/Documents/Unreal Projects/MyProject/Plugins/GestureTrackerVR/Source/GestureTrackerVR/Private/RecordGesture.cpp(35): error C4458: declaration of ‘resolution’ hides class member
1> c:\users\brandon\documents\unreal projects\myproject\plugins\gesturetrackervr\source\gesturetrackervr\private\Gesture.h(42): note: see declaration of ‘Gesture::resolution’
1>C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\VC\VCTargets\Microsoft.MakeFile.Targets(44,5): error MSB3075: The command ““C:\Program Files\Epic Games\UE_4.24\Engine\Build\BatchFiles\Build.bat” MyProjectEditor Win64 Development -Project=“C:\Users\Brandon\Documents\Unreal Projects\MyProject\MyProject.uproject” -WaitMutex -FromMsBuild” exited with code 5. Please verify that you have sufficient rights to run this command.
1>Done building project “MyProject.vcxproj” – FAILED.
@wandbrandon Gotcha, those can all be fixed up really quick. I’ll send Epic an update today but it usually takes them a few days to publish it.
EDIT: Just submitted the update to Epic. If you want to get it building yourself until they push it out all you need to do is change those parameter names on lines 30 and 35 in RecordGesture.cpp from startLocation, rotator, etc to inStartLocation, inRotator, etc and it should compile properly.
In my tests continuous recognition was working same as always in 4.24. If you’re still having issues I could probably help you troubleshoot with a little more detail about what you’ve set up.
The problem was that (in my project 4.23) the Play and Continuous was assigned to one controller button, in 4.24 - this no longer works. Reassigned to another button and now works fine
I’m developing on the Oculus Quest, and due to the need to package each run-through, I am setting up VR gestures on a different project, then manually migrating the save file over to the VR Project folder. All of that works well. The problem is that in the new VR project folder, when I try to load the gesture (in editor, or as a non-asset package), I am always getting the load failed branch of the blueprint.
Hello, I’m trying to compile my project which use the hand gesture. I’m using UE4 4.19.2. The problem is that this that due to this plug in, when i compile, it request the 32 bit Tool chain. I tried to install all Visual studio 2019 sdk bud nothing doesn’t work…
Any idea?
LogSlate: FSceneViewport::OnFocusLost() reason 0
LogSlate: Took 0.003781 seconds to synchronously load lazily loaded font ‘…/…/…/Engine/Content/Slate/Fonts/DroidSansMono.ttf’ (77K)
UATHelper: Packaging (Windows (64-bit)): ERROR: No 32-bit compiler toolchain found in C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.exe
PackagingResults: Error: No 32-bit compiler toolchain found in C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\cl.exe
UATHelper: Packaging (Windows (64-bit)): Took 15,1089762s to run UnrealBuildTool.exe, ExitCode=5
UATHelper: Packaging (Windows (64-bit)): ERROR: Command failed (Result:5): C:\Program Files\Epic Games\UE_4.19\Engine\Binaries\DotNET\UnrealBuildTool.exe MyProject2 Win64 Development -Project=“C:\Users\inscape\Documents\Unreal Projects\MyProject2\MyProject2.uproject” -generatemanifest -nobuilduht -xgeexport “C:\Users\inscape\Documents\Unreal Projects\MyProje
ct2\MyProject2.uproject” -NoUBTMakefiles -remoteini=“C:\Users\inscape\Documents\Unreal Projects\MyProject2” -skipdeploy -NoHotReload. See logfile for details: ‘UnrealBuildTool-2020.04.10-12.29.10.txt’
UATHelper: Packaging (Windows (64-bit)): (see C:\Users\inscape\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Program+Files+Epic+Games+UE_4.19\UAT_Log.txt for full exception trace)
PackagingResults: Error: Command failed (Result:5): C:\Program Files\Epic Games\UE_4.19\Engine\Binaries\DotNET\UnrealBuildTool.exe MyProject2 Win64 Development -Project=“C:\Users\inscape\Documents\Unreal Projects\MyProject2\MyProject2.uproject” -generatemanifest -nobuilduht -xgeexport "C:\Users\inscape\Documents\Unreal Projects\MyProject2\MyProject2.uproject
" -NoUBTMakefiles -remoteini=“C:\Users\inscape\Documents\Unreal Projects\MyProject2” -skipdeploy -NoHotReload. See logfile for details: ‘UnrealBuildTool-2020.04.10-12.29.10.txt’
UATHelper: Packaging (Windows (64-bit)): AutomationTool exiting with ExitCode=5 (5)
UATHelper: Packaging (Windows (64-bit)): BUILD FAILED
PackagingResults: Error: Unknown Error
First of all, this plugin is incredibly helpful! Thank you so much for making it! I’ve reached a roadblock though.
I have multiple gestures associated with the same command (UP can be a flick of the wrist up or a long arm movement) and I want to check for multiple commands (up, forward, side). I assume I need to save all the commands into the same file in which case I want to check that the predicted gesture has a specific command name associated with it. Is there a way to get the name of the predicted gesture without ending recognition? I’m trying to sense cases where both hands are moving up simultaneously. If not, is there a way to print out the array of gestures and their names so I can know what indexes are associated with what name?
Am I just being stupid? Does it make more sense to make a new Gesture Tracker for each kind of command (upGestureTracker, forwardGestureTracker)? I would really appreciate any advice anyone could offer.
Bind to the onGestureRecognized event to check for recognition before FinishRecognition is called, you can get the name from that. You could have a bunch of different save files for different gesture sets, you only need multiple gesture trackers to track both hands at once, you likely wouldn’t want to make one for each gesture. I would just save all the gestures you want to a set, one for each. Give the up gestures all the same name and check if you triggered a gesture with that name on both hands within some time frame.