# TensorFlow

**URL:** <https://forums.unrealengine.com/t/tensorflow/100206>\
**Category:** Game Development\
**Tags:** unreal-engine\
**Created:** [September 21, 2017, 7:23pm UTC](https://forums.unrealengine.com/t/tensorflow/100206 "2017-09-21T19:23:13Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [September 21, 2017, 7:23pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/1 "2017-09-21T19:23:13Z")

</div>

[](https://i.imgur.com/qn73w1u.png)

Hello devs!

For the past couple of months I’ve been working on bringing [TensorFlow](https://www.tensorflow.org/) to UE4, enabling the use of state of the art machine learning in our projects. The plugin has reached something I think is ready to try out, but it is very much a work in progress so please check it out and give some feedback if you can!

**Download**  
_Plugin_]([Releases · getnamo/tensorflow-ue4 · GitHub](https://github.com/getnamo/tensorflow-ue4/releases))

[https://img.shields.io/github/release/getnamo/tensorflow-ue4/all.svg](https://img.shields.io/github/release/getnamo/tensorflow-ue4/all.svg)

_Examples Project_]([Releases · getnamo/tensorflow-ue4-examples · GitHub](https://github.com/getnamo/tensorflow-ue4-examples/releases))

[https://img.shields.io/github/release/getnamo/tensorflow-ue4-examples/all.svg](https://img.shields.io/github/release/getnamo/tensorflow-ue4-examples/all.svg)

**Plugin Variants**  
Embedded- [GitHub - getnamo/tensorflow-ue4: TensorFlow plugin for Unreal Engine 4](https://github.com/getnamo/tensorflow-ue4)  
Remote - [GitHub - getnamo/machine-learning-remote-ue4: Machine Learning plugin for the Unreal Engine, encapsulating calls to remote python servers running e.g. Tensorflow/Pytorch.](https://github.com/getnamo/machine-learning-remote-ue4)  
Native - [GitHub - getnamo/tensorflow-native-ue4: Tensorflow Plugin for Unreal Engine using C API for inference focus.](https://github.com/getnamo/tensorflow-native-ue4)

**Limitations**  
Tensorflow-ue4 is Windows platform only. If you want to use a different platform for the client use the remote variant: [GitHub - getnamo/machine-learning-remote-ue4: Machine Learning plugin for the Unreal Engine, encapsulating calls to remote python servers running e.g. Tensorflow/Pytorch.](https://github.com/getnamo/machine-learning-remote-ue4). Additional work is being done to widen backend support: [Refactor to support Native and Remote variants · Issue #53 · getnamo/tensorflow-ue4 · GitHub](https://github.com/getnamo/tensorflow-ue4/issues/53)

**Resources**  
Latest documentation is always available at the github repositories.

_Plugin_

> **[GitHub - getnamo/tensorflow-ue4: TensorFlow plugin for Unreal Engine 4](https://github.com/getnamo/tensorflow-ue4)**
>
> TensorFlow plugin for Unreal Engine 4. Contribute to getnamo/tensorflow-ue4 development by creating an account on GitHub.

_Examples Project_

> **[GitHub - getnamo/tensorflow-ue4-examples: Drag and drop Unreal Engine...](https://github.com/getnamo/tensorflow-ue4-examples)**
>
> Drag and drop Unreal Engine TensorFlow examples repository - GitHub - getnamo/tensorflow-ue4-examples: Drag and drop Unreal Engine TensorFlow examples repository

**Examples - Mnist**

Made a simple mnist recognition example.

[![](https://d3kjluh73b9h9o.cloudfront.net/original/4X/6/d/9/6d9955109057cebc12a9e011102cab7b8454eb1a.gif) ](https://i.imgur.com/kvsLXvF.gif)

_Basic MNIST softmax classifier trained on begin play with sample training inputs streamed to the editor during training. When fully trained, UTexture2D (1-3) samples are tested for prediction._

you can even use your browser on your phone or desktop to draw some shapes to try recognition

[![](https://d3kjluh73b9h9o.cloudfront.net/original/4X/3/b/3/3b34ec58e48a516637ec78f4cd9013b7f475b5ea.gif) ](https://i.imgur.com/TAV4Rie.gif)

**Examples - Basic adding in TensorFlow**  
Want just the basics? check out the basic.umap map which uses [addExample.py](https://github.com/getnamo/tensorflow-ue4-examples/blob/master/Content/Scripts/addExample.py) to add float array together. This is a good entry point to see how to pass UE4 blueprint data into and out of python tensorflow.

[![](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/1/a/21a241556035e746370902e4684b19b687d3dfb7.png) ](https://i.imgur.com/I50IQ8h.png)

**Examples - Upcoming Pong with QLearning AI**

Still training the pong AI with sped up time dilation, but you can track updates in the [qlearn branch of the examples repository](https://github.com/getnamo/tensorflow-ue4-examples/tree/qlearn).

[![](https://d3kjluh73b9h9o.cloudfront.net/original/4X/7/2/e/72e38c21f4be288ff614c07d0120a8ae8c8641e2.gif) ](https://i.imgur.com/nsrsW7K.gif)

Let me know what you think and what machine learning examples you’d like to see in the future!

---

<div class="post-metadata">

**Author:** ![n00854180t](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@n00854180t](https://forums.unrealengine.com/u/n00854180t)\
**Post date:** [September 21, 2017, 9:37pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/2 "2017-09-21T21:37:09Z")

</div>

@getnamo - This is awesome!

There are certain really cool projects that use TF that I’d love to be able to integrate into a game, e.g., Google’s NLP parser SyntaxNet.

Training a net to differentiate a person stepping vs just leaning in VR would also be really cool (though there’s a nice asset for doing this already on the marketplace VR Footstep Plugin), and allow people to use full body avatars in VR without having to have extra trackers other than hands and head.

---

<div class="post-metadata">

**Author:** ![duyth](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@duyth](https://forums.unrealengine.com/u/duyth)\
**Post date:** [September 21, 2017, 10:51pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/3 "2017-09-21T22:51:42Z")

</div>

Cool stuff  
Would be awesome if we could use this to Train enemy AI in a RPG game /or enemies in RTS game and then save the progress

---

<div class="post-metadata">

**Author:** ![Mr\_SquarePeg](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@Mr\_SquarePeg](https://forums.unrealengine.com/u/Mr_SquarePeg)\
**Post date:** [September 22, 2017, 12:34am UTC](https://forums.unrealengine.com/t/tensorflow/100206/4 "2017-09-22T00:34:36Z")

</div>

This is awesome!

---

<div class="post-metadata">

**Author:** ![oakisnotree](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@oakisnotree](https://forums.unrealengine.com/u/oakisnotree)\
**Post date:** [September 30, 2017, 9:03am UTC](https://forums.unrealengine.com/t/tensorflow/100206/5 "2017-09-30T09:03:04Z")

</div>

> [@duyth](#):
>
> Cool stuff  
> Would be awesome if we could use this to Train enemy AI in a RPG game /or enemies in RTS game and then save the progress

You actually can

---

<div class="post-metadata">

**Author:** ![oakisnotree](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@oakisnotree](https://forums.unrealengine.com/u/oakisnotree)\
**Post date:** [July 19, 2018, 1:58pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/6 "2018-07-19T13:58:15Z")

</div>

I am wondering if the new editor scripting capabilities can be used to directly integrate tensorflow into our projects?

---

<div class="post-metadata">

**Author:** ![RoxyRypler](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@RoxyRypler](https://forums.unrealengine.com/u/RoxyRypler)\
**Post date:** [August 8, 2018, 8:42am UTC](https://forums.unrealengine.com/t/tensorflow/100206/7 "2018-08-08T08:42:33Z")

</div>

It would be cool to see an LSTM for sequence learning in this! enemy predicting the players actions and acting on it.

---

<div class="post-metadata">

**Author:** ![IOchair](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@IOchair](https://forums.unrealengine.com/u/IOchair)\
**Post date:** [September 4, 2018, 6:53am UTC](https://forums.unrealengine.com/t/tensorflow/100206/8 "2018-09-04T06:53:35Z")

</div>

Just curious: Can this plugin work after packaging?

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [September 10, 2018, 4:26pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/9 "2018-09-10T16:26:11Z")

</div>

Packaging support is a known missing feature. Progress is tracked here: [Packaging Support · Issue #16 · getnamo/tensorflow-ue4 · GitHub](https://github.com/getnamo/tensorflow-ue4/issues/16), contributions welcome 🙂

---

<div class="post-metadata">

**Author:** ![HaBe2305](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/habe2305/32/1069784_2.png) [@HaBe2305](https://forums.unrealengine.com/u/HaBe2305)\
**Post date:** [September 16, 2018, 8:08pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/10 "2018-09-16T20:08:23Z")

</div>

First of all, thank you very much @getnamo for your tremendous amount of work you’re giving away basically for free (the couple of bucks on the marketplace are merely symbolic in this regard). I’ve started to play with Reinforcement Learning and tensorflow using Unity and the Learning Agents preview they provide, but now have switched to Unreal because of - well, many reasons.

So already working with tensorflow, I was and still am excited that the Unreal environment also has integration already. Big thanks again for making this possible!

I also was used to having tensorboard for better understanding what is going on with my learning agent. I’ve tried for a day now to get tensorboard running (writing out logs with the simple MNIST example from [Beginnen Sie mit TensorBoard &nbsp;|&nbsp; TensorFlow](https://www.tensorflow.org/guide/summaries_and_tensorboard) worked flawlessly), but I’m totally lost now, also because I’m just beginning writing python code, coming from a C++ and C# background. Have you tested running tensorboard already and could kindly give some guideline or tips how to make it work?

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [September 22, 2018, 8:01pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/11 "2018-09-22T20:01:24Z")

</div>

> [@HaBe2305](#):
>
> First of all, thank you very much @getnamo for your tremendous amount of work you’re giving away basically for free (the couple of bucks on the marketplace are merely symbolic in this regard). I’ve started to play with Reinforcement Learning and tensorflow using Unity and the Learning Agents preview they provide, but now have switched to Unreal because of - well, many reasons.
> 
> So already working with tensorflow, I was and still am excited that the Unreal environment also has integration already. Big thanks again for making this possible!
> 
> I also was used to having tensorboard for better understanding what is going on with my learning agent. I’ve tried for a day now to get tensorboard running (writing out logs with the simple MNIST example from [https://www.tensorflow.org/guide/sum…nd\_tensorboard](https://www.tensorflow.org/guide/summaries_and_tensorboard) worked flawlessly), but I’m totally lost now, also because I’m just beginning writing python code, coming from a C++ and C# background. Have you tested running tensorboard already and could kindly give some guideline or tips how to make it work?

[https://github.com/getnamo/tensorflo…SimpleBoard.py](https://github.com/getnamo/tensorflow-ue4-examples/blob/master/Content/Scripts/mnistSimpleBoard.py) is the only experiment I did with tensorboard. My understanding is that you have to run tensorboard separately to host the webpage and then using something like the above, you can feed in all the data it needs for visualization. There isn’t anything in the plugin currently to bring that visualization into ue4, but you could use something like [GitHub - ashea-code/BLUI: Rich HTML UI engine for UE4](https://github.com/AaronShea/BLUI) or the inbuilt chromium to display the webpage inside ue4. If you get something working consider contributing a simple example to [GitHub - getnamo/TensorFlow-Unreal-Examples: Drag and drop Unreal Engine TensorFlow examples repository.](https://github.com/getnamo/tensorflow-ue4-examples/)

---

<div class="post-metadata">

**Author:** ![HaBe2305](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/habe2305/32/1069784_2.png) [@HaBe2305](https://forums.unrealengine.com/u/HaBe2305)\
**Post date:** [September 22, 2018, 8:45pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/12 "2018-09-22T20:45:52Z")

</div>

> [@getnamo](#):
>
> [https://github.com/getnamo/tensorflo…SimpleBoard.py](https://github.com/getnamo/tensorflow-ue4-examples/blob/master/Content/Scripts/mnistSimpleBoard.py) is the only experiment I did with tensorboard. My understanding is that you have to run tensorboard separately to host the webpage and then using something like the above, you can feed in all the data it needs for visualization. There isn’t anything in the plugin currently to bring that visualization into ue4, but you could use something like [GitHub - ashea-code/BLUI: Rich HTML UI engine for UE4 · GitHub](https://github.com/AaronShea/BLUI) or the inbuilt chromium to display the webpage inside ue4. If you get something working consider contributing a simple example to [GitHub - getnamo/TensorFlow-Unreal-Examples: Drag and drop Unreal Engine TensorFlow examples repository. · GitHub](https://github.com/getnamo/tensorflow-ue4-examples/)

Now that you say it, this absolutely makes sense. I tried to run tensorboard from inside UE, the idea that I easily could run it from a standalone python console as well on the logs didn’t come to my mind at that time. Thanks for the enlightenment.

I’ll experiment with the inbuilt chromium browser first. I wanted to adopt the workflow and setup Unity uses (external python distro via Anaconda) anyways since they have some nice goodies in their SDK like curiosity reinforced learning, circulum learning, imitation learning etc, and I’ll certainly contribute it to your repo as soon as it’s stable.  
Do you prefer pull requests from a branch, a fork or another way?

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [September 23, 2018, 9:32pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/13 "2018-09-23T21:32:38Z")

</div>

> [@HaBe2305](#):
>
> Now that you say it, this absolutely makes sense. I tried to run tensorboard from inside UE, the idea that I easily could run it from a standalone python console as well on the logs didn’t come to my mind at that time. Thanks for the enlightenment.
> 
> I’ll experiment with the inbuilt chromium browser first. I wanted to adopt the workflow and setup Unity uses (external python distro via Anaconda) anyways since they have some nice goodies in their SDK like curiosity reinforced learning, circulum learning, imitation learning etc, and I’ll certainly contribute it to your repo as soon as it’s stable.  
> Do you prefer pull requests from a branch, a fork or another way?

Sounds good :). A pull request on master is fine, I can always retarget it to a working branch if it breaks too much

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [October 23, 2018, 5:53am UTC](https://forums.unrealengine.com/t/tensorflow/100206/14 "2018-10-23T05:53:58Z")

</div>

_Recent update_:  
With the v0.10.1 ([Release v0.10.1 for UE4.20 · getnamo/tensorflow-ue4 · GitHub](https://github.com/getnamo/tensorflow-ue4/releases/tag/0.10.1)) release the plugin now supports packaging. Let me know if there are any issues with this.

---

<div class="post-metadata">

**Author:** ![anonymous\_user\_32498035](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@anonymous\_user\_32498035](https://forums.unrealengine.com/u/anonymous_user_32498035)\
**Post date:** [October 25, 2018, 11:14am UTC](https://forums.unrealengine.com/t/tensorflow/100206/15 "2018-10-25T11:14:20Z")

</div>

> [@getnamo](#):
>
> _Recent update_:  
> With the v0.10.1 ([Release v0.10.1 for UE4.20 · getnamo/TensorFlow-Unreal · GitHub](https://github.com/getnamo/tensorflow-ue4/releases/tag/0.10.1)) release the plugin now supports packaging. Let me know if there are any issues with this.

Have you released the pong game yet? I’m currently developing a simple 2d game with a stick that tries to avoid enemies that are moving in the environment by pulling up/down the stick.

---

<div class="post-metadata">

**Author:** ![anonymous\_user\_d8ab8b881](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@anonymous\_user\_d8ab8b881](https://forums.unrealengine.com/u/anonymous_user_d8ab8b881)\
**Post date:** [January 13, 2019, 8:53pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/16 "2019-01-13T20:53:28Z")

</div>

This is incredibly cool coding ! Can one use ue4 node editor as general c++ visual coding platform, by following this gourgeous implementation ? Is it basically about linking dll functions to the blueprint nodes ?

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [January 23, 2019, 6:38am UTC](https://forums.unrealengine.com/t/tensorflow/100206/17 "2019-01-23T06:38:52Z")

</div>

> [@Metusalemski](#):
>
> This is incredibly cool coding ! Can one use ue4 node editor as general c++ visual coding platform, by following this gourgeous implementation ? Is it basically about linking dll functions to the blueprint nodes ?

The unreal side for communicating data is typically done in blueprint (node editor) via a \*TensorflowComponent \*and the machine learning is done in python (which is what tensorflow uses so you can copy paste examples).

> [@anonymous\_user\_32498035](#):
>
> Have you released the pong game yet? I’m currently developing a simple 2d game with a stick that tries to avoid enemies that are moving in the environment by pulling up/down the stick.

Implemented it in a separate branch of the example project: [GitHub - getnamo/TensorFlow-Unreal-Examples at qlearn](https://github.com/getnamo/tensorflow-ue4-examples/tree/qlearn). All the pieces are there, it’s just missing a trained network to work in theory.

---

<div class="post-metadata">

**Author:** ![SertacOgan](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/sertacogan/32/970490_2.png) [@SertacOgan](https://forums.unrealengine.com/u/SertacOgan)\
**Post date:** [January 23, 2019, 1:22pm UTC](https://forums.unrealengine.com/t/tensorflow/100206/18 "2019-01-23T13:22:08Z")

</div>

This is awesome!

---

<div class="post-metadata">

**Author:** ![envenger](https://d3kjluh73b9h9o.cloudfront.net/original/4X/2/8/5/28502927f1d97ba1051b3b47ccfe17233bf41ef0.png) [@envenger](https://forums.unrealengine.com/u/envenger)\
**Post date:** [March 9, 2019, 9:36am UTC](https://forums.unrealengine.com/t/tensorflow/100206/19 "2019-03-09T09:36:27Z")

</div>

Is tensorflow 2.0 supported and would I need to change something to get it download by pip?

---

<div class="post-metadata">

**Author:** ![getnamo](https://d1ap1mz92jnks1.cloudfront.net/user_avatar/forums.unrealengine.com/getnamo/32/1194624_2.png) [@getnamo](https://forums.unrealengine.com/u/getnamo)\
**Post date:** [March 11, 2019, 1:13am UTC](https://forums.unrealengine.com/t/tensorflow/100206/20 "2019-03-11T01:13:17Z")

</div>

> [@envenger](#):
>
> Is tensorflow 2.0 supported and would I need to change something to get it download by pip?

In theory just changing _“tensorflow”: “1.10.0”_, to \*“2.0.0” \*inside [TensorFlow-Unreal/Content/Scripts/upymodule.json at master · getnamo/TensorFlow-Unreal · GitHub](https://github.com/getnamo/tensorflow-ue4/blob/master/Content/Scripts/upymodule.json) should do what you’re looking for, but there may be gremlins in the new tf API, kindly open a github issue with details ([Issues · getnamo/TensorFlow-Unreal · GitHub](https://github.com/getnamo/tensorflow-ue4/issues)) if you run into problems.

[Next page](https://forums.unrealengine.com/t/tensorflow/100206.md?page=2)
