For the past couple of months I’ve been working on bringing TensorFlow to UE4, enabling the use of state of the art machine learning in our projects. The plugin has reached something I think is ready to try out, but it is very much a work in progress so please check it out and give some feedback if you can!
Remote - https://github.com/getnamo/machine-learning-remote-ue4
Native - https://github.com/getnamo/tensorflow-native-ue4
Tensorflow-ue4 is Windows platform only. If you want to use a different platform for the client use the remote variant: https://github.com/getnamo/machine-learning-remote-ue4. Additional work is being done to widen backend support: https://github.com/getnamo/tensorflow-ue4/issues/53
Latest documentation is always available at the github repositories.
Examples - Mnist
Made a simple mnist recognition example.
Basic MNIST softmax classifier trained on begin play with sample training inputs streamed to the editor during training. When fully trained, UTexture2D (1-3) samples are tested for prediction.
you can even use your browser on your phone or desktop to draw some shapes to try recognition
Examples - Basic adding in TensorFlow
Want just the basics? check out the basic.umap map which uses addExample.py to add float array together. This is a good entry point to see how to pass UE4 blueprint data into and out of python tensorflow.
Examples - Upcoming Pong with QLearning AI
Still training the pong AI with sped up time dilation, but you can track updates in the qlearn branch of the examples repository.
Let me know what you think and what machine learning examples you’d like to see in the future!