As far as I remember, the struct observation simply concatenates the child elements, so the neural network should end up the same or very similar in your example.
Hi,
I think the course leaves out an important detail in the IL part: the trainer file needs to be set to ātrain_behavior_cloningā in the trainer process settings.
Otherwise, the python prcoess will try to run the default PPO trainer, and the ULearningAgentsImitationTrainer will not have sent the correct JSON config.
For Google, and for the next person trying to fix this, the subprocess will throw this error.
LogLearning: Display: Sending config signalā¦
LogLearning: Display: ImitationTrainer_8: Imitation Training Started
LogLearning: Display: ImitationTrainer_8: Sending / Receiving initial policyā¦
LogLearning: Display: Subprocess: Traceback (most recent call last):
LogLearning: Display: Subprocess: File āC:\Epic Games\UE_5.5\Engine\Plugins\Experimental\LearningAgents\Content\Python\train.pyā, line 53, in
LogLearning: Display: Subprocess: train(communicator.config, communicator)
LogLearning: Display: Subprocess: File āC:\Epic Games\UE_5.5\Engine\Plugins\Experimental\LearningAgents\Content\Python\train_ppo.pyā, line 30, in train
LogLearning: Display: Subprocess: ppo_config = config[āPPOSettingsā]
LogLearning: Display: Subprocess: ~~~~~~^^^^^^^^^^^^^^^
LogLearning: Display: Subprocess: KeyError: āPPOSettingsā
2 Likes
Just went through the course and everything was working until I got to the imitation training. For some reason, it canāt find the python executable:
LogLearning: Error: ImitationTrainer_0: Canāt find Python executable ā../../../../../../Unreal/LearningToDrive/LearningToDrive/Intermediate/PipInstall/Scripts/python.exeā.
BP_SportsCarManager is working fine, and able to spawn the python process. Both of them have the same settings in āTrainer Process Settingsā and the trainer file name has been changed to: ātrain_behavior_cloningā for imitation trainer.
What could be causing this error message? And why would it happen for imitation training only?
This is using 5.5.4.
Could somebody describe the available model/s?
Thank you so much! I got an inexplicable error here. It makes me unable to proceed. 
@Deathcalibur Any notable updates to Learning Agents we should be aware of in 5.6?
The big news is we updated the python so that itās possible to scale out training to ~100s of game processes in parallel.
Hereās the list:
-
Multi-Processed Training:
We are able to easily spawn X number of Unreal processes and have them all communicating with a single python process to be able to improve training throughput. -
Easy Tensorboard Installation
Simply use the new āTensorboardā plugin - no longer need to find the python env and manually install it via pip -
MLflow
Supports MLflow as an alternative to Tensorboard -
Shared Memory on Mac
-
Action Modifiers
Ability to modify the actions based on context. For example, mask invalid actions.
I donāt believe there are major breaking change with this tutorial and 5.6. Iāve been pretty busy working on a project so I havenāt taken the time to post 5.6 tutorial, which Iām sure is unfortunately confusing for some.
Thanks! Let me know if there are questions.
1 Like
Iām not sure if this is helpful or not, I follow Joseph Suarez on X. He seems to be doing a lot of research into RL and how to make it faster from the ground up. Iām curious if any of that research could be of use to the Epic team or if you are already aware of the work. Anyhow, here is a recent article of his 
Hopefully itās of some use.
1 Like
Thanks for the update @Deathcalibur !
Could you provide some pointers on how to get Multi-Processed Training up and running?
1 Like
Hi, I followed the tutorial, and it worked well. Then I extended it by adding custom logic for rotating at the start and stopping at the end of a spline. However, before reaching the maximum reward, the agents seem to āforgetā how to rotate. Iāve tried many adjustments (reward tuning, observations, parameters), but nothing has helped so far.
Any help would be appreciated!
Is there a 5.6 version of the tutorial ?
I canāt compile the GatherAgentObservation function in 5.6.
The ObsActor seems to be the problem for me. I tried to create it from the BP nodes (ācreate variableā, āpromote to variableā, then manually.
Anyone had it working in 5.6 ?
Thanks
1 Like
I havenāt updated the tutorial because I believe it should just work (could be wrong though). Itās a lot of effort to update the tutorial unfortunately and Iām rather busy. If there is more confirmed problems, then I might try to squeeze it in.
If you post a screenshot or something I might be able to help you figure out whatās wrong.
@FlameTheory
Take a look at this. Itās not really a turn crank solution. Mainly giving you the command line args needed to get setup with Docker.
Scaling Out with Learning Agents Tutorial.pdf (81.8 KB)
@tomhalpin8 I follow Joseph on X. His stuff is cool but I think most of it is not super applicable to UE games.
2 Likes
That was very helpful thank you!
I now have this working on macOS without the need for Docker. Here are the steps I took:
- Make a new project folder
- Copy over all the files from
/Users/Shared/Epic Games/UE_5.6/Engine/Plugins/Experimental/LearningAgents/Content/Pythonand/Users/Shared/Epic Games/UE_5.6/Engine/Plugins/Experimental/NNERuntimeBasicCpu/Content/Python - Create a new virtual environment:
python3 -m venv venv - Activate the venv:
source venv/bin/activate - Install numpy, tensorboard, torch, torchvision, torchaudio:
pip install torchetc - Start the server with:
python3 train.py Training -l --nne-cpu-path nne_runtime_basic_cpu.py Socket 127.0.0.1:48491 output - Make sure your UE project is setup to use sockets and external training
- Spawn as many instances as you like in new terminal windows with:
/Users/Shared/Epic\ Games/UE_5.6/Engine/Binaries/Mac/UnrealEditor -Project=/Path/To/Your/Game.uproject MapName -game -nullrhi -nosound
1 Like
Yes perfect! Glad to hear it works on Mac. I do something similar on my Windows PC. I wrote a script to spawn X UE processes.
1 Like
Thanks a lot for your answer and my apologies for the very late reply (couldnāt find the time to open UE5 during the week).
I finally found what didnāt work for me when copy pasting the Gather Agent Observation function.
When you automatically create the ObsActor variable (right click on āsetā node, or promote to variables in āgetā nodes), itās created with the type āobject referenceā, which is not compatible with all the nodes needing it.
So i had to manually change its type to āactor object referenceā, then it compiled.
1 Like