I am very interested by the push to use Deep Learning architectures in UE. Clearly, the ML Adapter plugin is meant to use UE as an environment (in Reinforcement Learning terms), so that standard RL algorithms can be trained on them.
I would like to know if we could have a very basic program where this is put in action. I beleve this would consist of an UE minigame like CartPole with a single agent, a few actuators and some function to output the game state as some vector of data and/or some image. And a jupyter/python script that uses OpenAI gym (or gymnasium) to train a RL algorithm on it. Which implies getting the game state, getting the possible actions the agent can take, and being able to “push” an action into UE.
This seems to be extremely promising, and I can’t wait to build on it, but I think the documentation is currently too hard to parse for an outsider to get a toy project running.
Thanks for your interest in ML Adapter. It’s great to hear that people are interested in deep learning in UE!
I haven’t written documentation/provided examples yet because I’m currently in-progress on exploring a completely different approach to using ML with UE. The alternative will be one where UE controls the ML training by setting it up and calling it directly. This is opposed to ML Adapter where the external training process is driving the training and interaction with the environment. This newer design feels a lot more natural for actually using ML as a game dev and has better performance vs the current approach is probably more natural for ML researchers.
Still exploring the design so no promises about the future.
Thanks for your patience! I am happy to share that we have pushed out the very first version of a new plugin, Learning Agents! This plugin is similar to ML Adapter but flips the design on its head a bit: instead of the python training process “controlling” Unreal, the Unreal process is in charge and controls the python one. You can learn a little more about it here: Learning Agents Introduction
We believe this design is much better on several axes including runtime performance, flexibility for future growth, and fits many more use cases. For example, it’s much easier to replace a module in a traditional AI behavior tree with a model created with Learning Agents, where ML Adapter would encourage you to replace your whole AI altogether.
The current design’s foundations are hopefully relatively stable, but the plugin is still experimental so some breaking changes may be needed. We mostly intend to add new observations and actions to hit common use-cases, but the AddFloatObservation and AddFloatAction can be used for almost anything if you’re willing to do more preprocessing work yourself. We are also currently working on adding comments to all the codebase as well as a developer course which will get you familiar with the plugin and walk you through an example of building and training your first agent.
One other note is that the current training and neural network support is relatively limited. We currently are supporting “vanilla” feed forward models and a PPO RL implementation, as well as a basic behavior cloning imitation model. We intend to expand these options greatly while the plugin is in the experimental state.
Please check it out if you are so inclined and feel free to contact us here on the forums or you can reach out to learning-agents@epicgames.com. Any feedback is welcome and you can have a huge impact on the future of ML in Unreal.
I’m the Mario of the repository and video you linked!
I hope the repo was in some way useful for you all.
Right now, I’m working on a new demo, where I’m using “ML Adapter”, and a more complex testing situation. I’ll let you know posting here when something is ready.
@Deathcalibur Thank you very much for the news of this new plug-in!
It’s a very interesting approach to this field!
I’m downloading the code right now to try it.
But, as I can see from the presentation, this plugin is oriented to support the creation of ML-driven elements inside unreal. And use those elements for games or other kinds of distributable unreal-based software.
While my approach, and what appears to be possible to streamline with “ML Adapter”, is different:
I come from the industrial simulation field, and my goal now is to use Unreal as a largely capable simulation environment, where let my agents live and try, piloted by external models.
Thats because my final goal here is to use the results of those training in other applications outside Unreal. Like real robots, digital twins, and so on.
What do you think? Is this a good approach of do you suggest a different one?
Thanks for all the work and support you and your team are doing!
