You can’t realistically use images since we don’t support CNNs in the inference runtime, nor do we currently provide any image observation objects. These are low priority for the time being, but technically they can be added in user space.
Great work Brendan! This is coming along nicely.
I was wondering, what kind of dependencies are there that are exclusive to UE5? Was wondering how hard it would be to port this to 4.27
I’m not sure since I haven’t tried. From what I recall, some of the observations related to splines depend on functions that don’t exist in earlier UE versions (but those observations could merely be commented out).
There might be issues with getting the correct python/pytorch setup, since we use the Python env built into Unreal.
If you try it and get it working, would be happy to hear.
Brendan
Do you know if there are any other packages that would allow me to do IL w/image data (in addition to articulation data). Using CV is essential for the robotics use case of my company…
I’m not aware. Sorry!
I will keep this use-case in mind.
Brendan
1 Like
Hey @Deathcalibur, is there an official roadmap for Learning Agents? Can you confirm any new features in 5.4?
Thanks!
2 Likes
Does anyone have any advice on delayed rewards? I’m training a boxer where an action might result in a reward around 0.35 seconds later due to the time taken for the punch to land. I can increase the tick rate, but 0.35 seconds is too high of a reaction time and results in extremely slow training.
Any help is appreciated, thanks
Been stuck for a while, don’t know where and how to start debugging, it’s a bit above my paygrade, any ideas would be welcome.
This is in my Trainer Manager
But none of the Events in the Trainer blueprint are triggering, and the Output Log repeats this:
[2024.04.12-04.06.08:257][732]LogLearning: Error: LearningTrainer: Setup not complete.
Are there any other errors or warnings near the start of the log? Usually you’ll find something there. Something happened during the call to SetupTrainer that caused it to fail, and then each call to RunTraining will not work.
Yup, found these a couple of dozen lines above the Trainer one
[124]LogLearning: Error: LearningPolicy: Neural Network Asset provided during Setup is incorrect size: Inputs and outputs don’t match what is required.
[124]LogLearning: Error: LearningTrainer: LearningPolicy’s Setup must be run before it can be used.
I did change the Observations and Actions completely but I assumed the ‘Reinitialize Polic Network’ would reset it.
I had to open the actual neural network Data Asset and click the Reset button
Thank for the help again!
1 Like
Good day, Im in 5.4p and there are alot of changes to the plugin, any resource avail yet for initial setup?
Hey everyone,
We just published a new course for the 5.4 version of Learning Agents. It mainly covers the driving scenario again, and we added extra tutorials which cover the FAQs from the past months.
Course outline
Introduction
Reinforcement Learning Tutorials
Imitation Learning Tutorials
Let us know what you think!
Thanks,
Brendan
5 Likes
Hello everyone,
I have a problem with inference, during the training task my agent moves at a really good speed, but during the inference phase, the speed is very slow.
do other people have this kind of problem?
Is the speed of the cars actually changing or merely the speed of the simulation? Training is used fixed time step so it could collect data faster than wall time. This is turned off at inference time so you’ll see the world at “regular” speed.
Or is the actual speedometer showing much slower speeds?
I forgot to mention that it’s not on the cars, but on a thirdperson agent, for a navigation task,
training is done at a good speed, but at the inference stage, the agent moves in slow motion.
1 Like
What function are you calling for the movement?
the basic one from unreal third person
Great to see the progress. Will try again in a few versions.
Will it be possible to use this in a shipped game in any way? Let’s say for some kind of limited online learning during actual gameplay.
1 Like
Currently you can do inference in a shipped game but you can’t really do training at runtime (technically you can but you have to have the Engine’s python env installed on the device where the UE process is running - we use this for headless training on a dev workstation for example).
For UE 5.5, I’m currently working on making training at shipping time more feasible by changing up how the UE process connects to the training process. It will be possible to run training on a server so you can, e.g. have an agent gathering data on any device and then sending this data up to a game server where you can train and then sync the model back down to the client device.
Ultra long-term we could maybe work out training within the UE process itself but this might never come so do not count on that.
Brendan
How feasible is to run training on replay data?
Like for instance with the Lyra sample, if I have a group of testers, they all sent me their replays.
And now I want to do some imitation learning with this gameplay data, locally on my machine. Would this be possible with the current version of the plugin?