Get familiar with Learning Agents: a machine learning plugin for AI bots. Learning Agents allows you to train your NPCs via reinforcement & imitation learning. It can be used to create game-playing agents, physics-based animations, automated QA bots, and much more!
https://dev.epicgames.com/community/learning/courses/kRm/unreal-engine-learning-agents-5-4
7 Likes
Physics Based Animations you say? 
I am very curious about that side of things!
Also thank you for the in-depth 5.4 Learning to Drive Tutorial!
3 Likes
Will version 5.4 train faster than 5.3, for example, 5.3 cpu usage may only be about 30%, will 5.4 support more cpu usage
3 Likes
I don’t know to be honest. We haven’t really looked into maximizing the CPU utilization yet. If you train with the headless instructions that we posted, it should be faster to gather observations from the environment.
For 5.5, we are working on re-writing the training API to be more flexible, i.e. not PPO/BC specific. We also have the ability to spawn multiple game processes to further parallel data rollouts, and have a training process hosted on another machine.
Brendan
4 Likes
First of all, thank you for the course. It is really easy to follow, and the changes from version 5.3 to 5.4 were really good for structuring the observations and rewards!
However, I have a question about the observations—I’m not sure if I’m understanding them correctly.
I tried to add obstacles to the track, and if a car collides with them, it should receive negative rewards to encourage avoiding the obstacles. I created an actor that produces overlaps and stores information about who collided with it.
Interactor:
- In the “specify agent observation” section, I added a location observation and constructed an array observation to add to the map containing the track and car observations. Then, I added a pin there called “Obstacles.”
- In the “gather agent observation” section, I added a pin to the sequence to gather the observations of the obstacles.
I didn’t change anything in the actions of the agent because I thought the actions should remain the same.
Trainer:
- I updated the reward function based on whether an obstacle was hit or not, giving -1 per obstacle hit by the agent.
- I also updated the reset function to return the obstacles to a clean state.
After these changes, I broke it, haha. Now, the score is not going above 0.85, no matter how long I let it run.
Could it be that my reward function or my observations are wrong? It would be nice to have an example of how to gather observations of objects on the track that are not part of the spline. I tried following the example of the robot with the gun and so on, but I don’t know if I’m doing something wrong.
In any case, thank you for your time in building the course. It was really easy to follow.
1 Like
Can you send a screenshot or two? It sounds fine but would be helpful to see.
Thanks!
1 Like
First of all thanks for the fast response!
Actually this are the obstacles, they are fix just to test if they are able to avoid something that i put in the track:
The obstacle actor:
Now the interactor:
Specify Agent obervations function:
I added tha part Obstacle observation + de entry on the map for the obstacles.
Gather Agent observations function
I added this as an step of the sequence
I added this to add the gathered observations on the sequence to the map
Now the trainer:
Gather Agent Reward function
I added this as part of a sequence to calculate when the agent hitted any of the obstacles and if it happens it add the negative reward.
Reset Agent Episode function
I added the reset of the obstacles to not mantain when a agent hitted an obstacle in the next iteration.
Also i modified the Reset function of the pawn to not fall in an obstacle when they are moved at the begining of the episode.
And this were my modifications, i was able to run it headless, using tensorboard etc… the only step that i need to do is restart from an snapshot to continue the training, but that is something that i can try later.
And thanks again for the fast response to the previous comment! I hope you find something on those pictures that im doing wrong and then i can improve!
At this part, when I paste the code. The Found Manager is not found and never mentioned.
I created the Variable manually and connected the new getters and setters. Which seems to work
Later at Learning Agents (5.4) | Course
in Perform Agent Action I assume there is a “create a local Var of Act Actor” missing.
At Reset Agent Episode It does not copy the Reset To Random On Spline Node.
When I add it manually it also looks different.
I gave up at the Begin Play Section
I wonder if people fixed everything on their own, or 5.4.1 changed so much.
1 Like
@Deathcalibur By any chance, now that Apple is opening up more of their Machine Learning efforts with MLX and Corenet GitHub repos, will the team be open to looking at possibly adding features to Unreal so we can get the Learning Agents and NNE to work on Mac?
At this part, when I paste the code. The
Found Manageris not found and never mentioned.
I’ll take a look and see if I can figure out what went wrong. Thanks for the feedback.
@tomhalpin8 I’ll ask the NNE team about Apple’s ML stuff.
For LA, the (seemingly) only reason its not working is because of a bug in UE’s shared memory code. We’re working on resolving this and also adding support for other communication protocols like TCP Sockets and HTTPS. We should be able to get Mac and Linux fully working for UE 5.5
Brendan
1 Like
Thank you for always getting back so quickly! I keep coming back to Learning Agents, one day it will finally click on how it works. I’m trying to figure out how I can use this to train a physics skeletal mesh, using torque on the joints to try and match a regular animated skeletal mesh. Hopefully train it to balance and walk purely through physics. The goal is to try and do something like this but staying entirely in Unreal Engine. I know it’s long shot for someone who doesn’t fully understand machine learning.
It’s a great goal to have! I would start with training some basics like moving to a goal, etc.
When I was first getting back into ML years ago, I found it very helpful to follow the Fast AI course: https://course.fast.ai/ It’s not related to game dev but Jeremy does great explanations of ML with a very programmer centric point of view. He talks about some math but its not a math-focused perspective.
2 Likes
Hey, I just started the course and I’m having this issue of PythonScriptPlugin error it gets stuck at 75% when I install Learning Agents 5.4.
It was actually downloading for a long time in the background. I just had to wait (long time) and its good now
1 Like
Hi, I’ve worked through the tutorial, but I’m getting an error when I get to the “training section”.
Any help would be greatly appreciated on this error?
Take a look at this thread: Can't find LearningAgents plugin Content
You seem to have the same issue.
I ran into this problem. It is because the reference ID of the TrackSpline variable is not the same as the one pasted in via the tutorial.
To fix this, you can simply go to your BP_SportsCarManager, right click, turn off Context Sensitive search for TrackSpline, and select the Get Track Spline node under the BP Sport Car Track Spline blueprint, then hook it up as expected and delete the old node.
I’m currently facing an issue where everything is hooked up and compiles, the cars spawn randomly across the track, but then … nothing happens. They don’t move, and then every 30s or so the termination function occurs and randomises their locations along the track again.
Will post a solve if I fix it.
How do you get “SET” node for Track Spline after Make Interactor in BP_SportsCarManager? What type is Track Spline supposed to be?
1 Like
Hello,
- Make sure the variable is set to the appropriate type and not the base class (or make sure to cast it to the appropriate type).
- Then, drag the pin from the trainer and search there.
Sometimes I have this minor struggle too and usually its because I forgot to set the variable to the correct type.
Let me know if that doesn’t work.
Brendan
1 Like
