No announcement yet.


  • Filter
  • Time
  • Show
Clear All
new posts

    [PLUGIN] TensorFlow

    Hello devs!

    For the past couple of months I've been working on bringing TensorFlow to UE4, enabling the use of state of the art machine learning in our projects. The plugin has reached something I think is ready to try out, but it is very much a work in progress so please check it out and give some feedback if you can!


    Examples Project

    Windows platform only atm. Consider contributing to add support for your platform!

    Latest documentation is always available at the github repositories.

    Examples Project

    Examples - Mnist

    Made a simple mnist recognition example.

    Basic MNIST softmax classifier trained on begin play with sample training inputs streamed to the editor during training. When fully trained, UTexture2D (1-3) samples are tested for prediction.

    you can even use your browser on your phone or desktop to draw some shapes to try recognition

    Examples - Basic adding in TensorFlow
    Want just the basics? check out the basic.umap map which uses to add float array together. This is a good entry point to see how to pass UE4 blueprint data into and out of python tensorflow.

    Examples - Upcoming Pong with QLearning AI

    Still training the pong AI with sped up time dilation, but you can track updates in the qlearn branch of the examples repository.

    Let me know what you think and what machine learning examples you'd like to see in the future!
    Plugins: TensorFlow - Client - ZipUtility - Leap Motion - Hydra - Myo - RealSense - CIM

    getnamo - This is awesome!

    There are certain really cool projects that use TF that I'd love to be able to integrate into a game, e.g., Google's NLP parser SyntaxNet.

    Training a net to differentiate a person stepping vs just leaning in VR would also be really cool (though there's a nice asset for doing this already on the marketplace VR Footstep Plugin), and allow people to use full body avatars in VR without having to have extra trackers other than hands and head.
    Last edited by n00854180t; 09-21-2017, 06:35 PM.
    Storyteller - An immersive VR audiobook player

    Dungeon Survival - WIP First person dungeon crawler with a focus on survival and environmental gameplay ala roguelikes


      Cool stuff
      Would be awesome if we could use this to Train enemy AI in a RPG game /or enemies in RTS game and then save the progress


        This is awesome!


          Originally posted by duyth View Post
          Cool stuff
          Would be awesome if we could use this to Train enemy AI in a RPG game /or enemies in RTS game and then save the progress
          You actually can