Get familiar with Learning Agents: a machine learning plugin for AI bots. Learning Agents allows you to train your NPCs via reinforcement & imitation learning. It can be used to create game-playing agents, physics-based animations, automated QA bots, and much more!
https://dev.epicgames.com/community/learning/courses/kRm/unreal-engine-learning-agents-5-4
5 Likes
Physics Based Animations you say? 
I am very curious about that side of things!
Also thank you for the in-depth 5.4 Learning to Drive Tutorial!
3 Likes
Will version 5.4 train faster than 5.3, for example, 5.3 cpu usage may only be about 30%, will 5.4 support more cpu usage
3 Likes
I don’t know to be honest. We haven’t really looked into maximizing the CPU utilization yet. If you train with the headless instructions that we posted, it should be faster to gather observations from the environment.
For 5.5, we are working on re-writing the training API to be more flexible, i.e. not PPO/BC specific. We also have the ability to spawn multiple game processes to further parallel data rollouts, and have a training process hosted on another machine.
Brendan
3 Likes
First of all, thank you for the course. It is really easy to follow, and the changes from version 5.3 to 5.4 were really good for structuring the observations and rewards!
However, I have a question about the observations—I’m not sure if I’m understanding them correctly.
I tried to add obstacles to the track, and if a car collides with them, it should receive negative rewards to encourage avoiding the obstacles. I created an actor that produces overlaps and stores information about who collided with it.
Interactor:
- In the “specify agent observation” section, I added a location observation and constructed an array observation to add to the map containing the track and car observations. Then, I added a pin there called “Obstacles.”
- In the “gather agent observation” section, I added a pin to the sequence to gather the observations of the obstacles.
I didn’t change anything in the actions of the agent because I thought the actions should remain the same.
Trainer:
- I updated the reward function based on whether an obstacle was hit or not, giving -1 per obstacle hit by the agent.
- I also updated the reset function to return the obstacles to a clean state.
After these changes, I broke it, haha. Now, the score is not going above 0.85, no matter how long I let it run.
Could it be that my reward function or my observations are wrong? It would be nice to have an example of how to gather observations of objects on the track that are not part of the spline. I tried following the example of the robot with the gun and so on, but I don’t know if I’m doing something wrong.
In any case, thank you for your time in building the course. It was really easy to follow.
1 Like
Can you send a screenshot or two? It sounds fine but would be helpful to see.
Thanks!
1 Like
First of all thanks for the fast response!
Actually this are the obstacles, they are fix just to test if they are able to avoid something that i put in the track:
The obstacle actor:
Now the interactor:
Specify Agent obervations function:
I added tha part Obstacle observation + de entry on the map for the obstacles.
Gather Agent observations function
I added this as an step of the sequence
I added this to add the gathered observations on the sequence to the map
Now the trainer:
Gather Agent Reward function
I added this as part of a sequence to calculate when the agent hitted any of the obstacles and if it happens it add the negative reward.
Reset Agent Episode function
I added the reset of the obstacles to not mantain when a agent hitted an obstacle in the next iteration.
Also i modified the Reset function of the pawn to not fall in an obstacle when they are moved at the begining of the episode.
And this were my modifications, i was able to run it headless, using tensorboard etc… the only step that i need to do is restart from an snapshot to continue the training, but that is something that i can try later.
And thanks again for the fast response to the previous comment! I hope you find something on those pictures that im doing wrong and then i can improve!
At this part, when I paste the code. The Found Manager is not found and never mentioned.
I created the Variable manually and connected the new getters and setters. Which seems to work
Later at Learning Agents (5.4) | Course
in Perform Agent Action I assume there is a “create a local Var of Act Actor” missing.
At Reset Agent Episode It does not copy the Reset To Random On Spline Node.
When I add it manually it also looks different.
I gave up at the Begin Play Section
I wonder if people fixed everything on their own, or 5.4.1 changed so much.
@Deathcalibur By any chance, now that Apple is opening up more of their Machine Learning efforts with MLX and Corenet GitHub repos, will the team be open to looking at possibly adding features to Unreal so we can get the Learning Agents and NNE to work on Mac?