A Gesture recognition plugin for Unreal Engine 4 allowing to record, detect and follow the progression of gestures from motion controllers in real-time with a high precision.
As a developer/designer, you only need to record one example per gesture. This can be done live in the game.
Once the template has been recorded, the listener will be able to detect when a gesture is started and follow its progression in real time.
You can freely add events based on the progression such as triggering a spell once the gesture is 95% done, or tick a spell for each 10% of gesture completed.
**The plugin is currently under development but the following core features are already functional:
- Simple component setup allowing fast integration in a project from BP or C++
- Offline and online reference gesture recording
- Only one reference gesture needed to teach the recognizer
- Allow the usage of 2D and 3D gestures
- Real time recognition with high accuracy and robustness
- Possibility to run multiple instance of the recognizer in parallel
You can test an early version(Vive only) here: https://goo.gl/XRNYTu
You can download the example project files on GitHub: GitHub - Deams51/UE4-Gesture-Recognition-Plugin: A 3D Gesture recognition plugin for Unreal Engine 4 allowing to record, detect and follow the progression of gestures from motion controllers in real-time with a high precision.
In this video you can see a gesture being recorded and then used to cast a spell:
Here, you can see an example of actions bound at run-time to gestures (binding a square to spawn a cube, a circle for a sphere…):
Projects using gestures
If you created a new VR experience using gestures, let us know and we will add it here for all to see!
This plugin would not have been possible without the previous work published by the HCI researchers over the years, and especially the following article by B. Caramiaux et al:
B. Caramiaux, N. Montecchio, A. Tanaka, F. Bevilacqua. Adaptive Gesture Recognition with Variation Estimation for Interactive Systems. ACM Transactions on Interactive Intelligent Systems (TiiS), 4(4), 18-51. December 2014
Feel free to post here or create an issue on GitHub.