Unity has launched a blog series focusing on game development using Machine learning and Deep Reinforcement Learning. I would really like to know how Unreal is supporting Game developers in implementing these technologies.
They aren’t.
From - vice president of AI and machine learning at Unity Technologies
The closest Unreal has anything to deep reinforcement learning is the tensorflow plugin.
but be warned any use of neural networks or reinforcement learning or even genetic algorithms require you to understand the maths behind the algorithms to be able to use them at a relative capacity.
Though unity isn’t doing deep reinforcement learning, they are doing genetic reinforcement learning which is quite a ways away from deep reinforcement learning.
Umm yea, if you have a masters in mathematical studies you might be able to make effective use of the algorithms. I tried jumping onto the AI and neural network train a while ago, and I jumped off because I couldn’t keep up with the advanced math. Really interesting stuff though. I have a lot of respect for the people who make those things work. Maybe that’s why Epic Games hasn’t set any priority to it. It isn’t a common skill that people have. They had to make blueprints for crying out loud for people who didn’t want to learn how to code (or didn’t have the time to learn). So I don’t see them backing this any time soon. I’m sure that data scientist could make some interesting use of UE4 if they plugged an AI into it though. That would be interesting.
If Unreal can create some APIs to link UE4 with libraries like Tensorflow, the complexity of training a non-player character (NPC) in a game would decrease a lot. Then, you wouldn’t need very heavy mathematics to create everything from scratch. If unity is seriously pursuing it and then I believe Unreal can also help us.
Honestly if someone can explain it well, you can understand everything you need to know at a high school / before any degree. Although, like you said mathematics is not a common skill, nor a strong skill in any sense. I know programmers who are terrible at math but can program really well, so it is possible to understand complex things without the math behind it.
I have huge respect for the developers in the car industry right now. They are making cars that can make it appear to predict accidents far before a human can react. look up some Tesla crash ‘prediction’ videos. while on the face it’s not doing what you’d expect. it is some insight into the future to come. (its not predicting crashes, it just seems like it b/c of driver assist) There’s some crazy powerful stuff in the future of ai and maths. we have seen ai do seemingly impossible things. I’ve done my most of my research in ai and have made a Deep reinforced genetic racing ai in ue4. (https://www.youtube.com/watch?v=NtsJf0ottIw) It’s crazy easy to get something moving. It’s insanely hard to get something that can be useful. I wish i knew how to teach better. because i think it’s a new way to think about things and id like to share it. not just for games but for life in general. but i only have so much time and i want to make games too lol. so probably left better off for someone else. besides they are smarter.
I don’t think adding links to libs like that are going to make making ai easier. I personally feel it increases the complexity allot. Go install all the dependencies of the tensorflow plugin linked above and check out the examples, maybe use it if you feel like it. but i know from experience. learning this can be tough and unforgiving and you feel like you understand it then you cant expand it or change it. or sometimes you just don’t have data to model and that is a problem. you can make data but how do you know what that means and how can that actually be useful. what does it mean to fall into a local minimum and how can you fix that? what activation function do you use? there is allot to it and much more. really not something so immediately usable and useful.
behavior trees are crazy powerful and i don’t feel they even been taken fully advantage of in games yet. They are far more understandable and immediately usable and useful. honestly i would start there. i truly think there still are great strides to be had in the field in addition to the more advanced ai’s. once it’s easier to have a black box to plug into then it will be worth it, but for now i really hope epic focuses on the engine itself.
If you really want to learn about this stuff Siraj Ravel on youtube is a great source. while not directly useful in ue4, the concepts can be. and combined with tensorflow ue4 plugin, the sky’s the limit.
https://www.youtube.com/channel/UCWN…TPmbKwht9FuE5A
I’ll be releasing more documentation very soon but just wanted to point out that my tensorflow plugin has matured and there is an example repository: GitHub - getnamo/tensorflow-ue4-examples: Drag and drop Unreal Engine TensorFlow examples reposito with basic working mnist recognition samples (from a basic softmax implementation to a Keras CNN written in vanilla tensorflow python) which can train straight in your editor in the background with no game thread penalty.
Then, after your network has been trained, you can test them on any UTexture2D, or go to Drawing Trivia e.g. on your phone or desktop browser and draw to recognize 0-9 digits. For better recognition I recommend using the Keras CNN script, but this may take 18min to train on a cpu (vs ~30sec on a gpu).
The plugin has full nvidia cuDNN gpu support and is self contained, grab an early release of the example project at https://github.com/getnamo/tensorflow-ue4-examples/releases/ (don’t forget to download the plugin and drag it into your downloaded project).
My aim here is to do all the boring integration bits so you only have to concentrate on learning tensorflow and then later implement those cool new smart features in your games. That said @SaxonRah is correct in that it may take some time for smart people to take research papers and open source repositories to convert them for use in UE4 before advanced examples become fully plug and play, but this should make that much easier than before.
More detailed tutorials will come soon 
@getnamo That plugin looks pretty awesome, although I’m struggling to understand how exactly it’s intended to be used. I have a basic understanding of machine learning, but no knowledge of the python environment that I understand now dominates ML research.
I’d suggest a high level overview of the plugin and example use cases would be a good starting point for documentation. Given that it’s implemented through an actor component, I’m assuming the idea is to support some kind of live learning in a packaged project? Is it geared towards image-based input or entirely agnostic in that respect? I’m wondering if it could potentially be used to train control systems by providing feedback data from live gameplay/PIE sessions.
Good suggestions, the plan is to have a video with a basic empty project to finished example run through so you can understand how it all comes together.
The current mnist examples use UTexture2D as input, but it’s entirely data agnostic. This is the full and current (1.3) tensorflow support with arbitrary input/output. To achieve this, it uses automatic struct json serialization as well as some convenience format conversions to communicate with the python layer. Even for images however I haven’t yet hit a performance block from json and it’s a huge boost to productivity to pass in near-arbitrary blueprint type data with no extra code. Typically you structure your input as e.g. a blueprint struct and pass it in, then on the python side you run it through your tensorflow graph and pass back the result as a python object, this then get’s automatically converted again and you can set a result bp struct from that data.
The examples provided are the easiest simple machine learning examples there are, equivalent to ‘hello world’ and don’t necessarily have practical use cases. They do however, provide a full implementation so you can dissect training, running your data through the graph for prediction, and how to convert data between python and unreal.
You are correct that the plugin is implemented as an actor component (with a python script) and it does indeed run in game worlds so you can have game feedback loops. You can also optionally train a network offline and run your inputs only through the trained network at runtime, skipping the training portion.
A final note: I knew no python before starting the plugin. I imagine with a small bit of programming knowledge most people can pick it up with no greater issue and the upside is you can copy paste other machine learning examples with little modification and they will run. Being a dynamic scripting language, you can also change code between play sessions and it will auto-update on play.
Thanks for the info, I’ll need to look into Tensorflow when I get some time and get up to speed on how it works. Thinking about it a bit more, I think I was mostly wondering if it’s possible to do learning where the evaluation aspect is live (for example, a measure of how well the controlled entity is maintaining stability in the live simulation) as opposed to by comparison with a precalculated set of training data. My limited ML experience was in neuro-evolution/genetic algorithms, so my understanding is probably skewed by that context; it may be something completely outside of what Tensorflow is designed for.
Automatic Json serialization sounds interesting, I’m going to take a look at that. It strikes me as bizarre that the only Json UE4 plugins I’ve seen just expose generic typed operations to blueprint, when the reflection system lends itself so well to automatically serializing any reflected object/struct.
As for Python, yep I agree it’s awesome (I was referring more to having no experience with ML in Python), and what the author of the UnrealEnginePython plugin has done is really impressive.
Tremendous potential for using Machine learning in unreal engine 4, recently published a Q – learning algorithm in the marketplace demonstrating how an AI character using reinforcement learning can solve a match to sample puzzle. This is common experiment to assess instrumental learning skills in animals.
Video available here: https://www.youtube.com/watch?v=Wy7HDj2igPo
And the Marketplace link: https://www.unrealengine.com/marketp…nce-q-learning
If there’s interest, will continue publishing machine learning experiments in UE4 and have made a simple blog to document some of this work.
I have gone through a lot of stuff but if you are interested in AI and Deep learning, i loved this one Bringing gaming to life with AI and deep learning – O’Reilly
have a go.
Theres a free open source plugin for UE called MindMaker AI that uses the OpenAI based protocol for deep learning with custom action and observation spaces. It currently works with a variety stable baselinesDeep RL algorithms including A2C, PPO, TD3 etc.
@getnamo thanks for writing the tensorflow plugin. I’d like to experiment with a tensorflow based model for generating music called Magenta (developed by google), but it seems that their api is python based, do you think your plugin could be used to expose the model ? Or is there a way to essentially run python in the background with the right libs installed ?
You likely want something like: GitHub - getnamo/machine-learning-remote-ue4: Machine Learning plugin for the Unreal Engine, encapsulating calls to remote python servers running e.g. Tensorflow/Pytorch. which lets you run computer local python or even remote server. It’s agnostic to actual ML libraries (e.g. pytorch, TF or even custom possible)