Learning Agents Imitation Learning Recording logic question

YoloSwaggerson
Wed, 04 Mar 2026 10:08:19 +0000
Programming & Scripting

Hello. I have a question about Imitation Learning Recording logic. So I basically went by “Learn to drive” LA tutorial. I am working on a stock UE 5.7.2.
I have an IL recording manager that in each update calls
ULearningAgentsController::RunController
ULearningAgentsRecorder::AddExperience

I also use a player imitator character as actions producer, so I manually play during IL recording sessions and create actions via ULearningAgentsActions::Make_X_Action.

and ULearningAgentsController::RunController calls
Interactor->GatherObservations();
EvaluateController();
Interactor->PerformActions();

THE PROBLEM:
And the function call “Interactor->PerformActions()” is a problem for me, because in my interactor PerformAgentAction_Implementation logic it is expected that certain components exist on agent actor (to be specific, components like “do X agent action” so that I don’t cast to a character class directly) and

1. My imitator character class doesn’t have these components because it is not intended to be a LA agent. So no components → need to add redundant null checks in the interactor

2. I don’t really understand why IL controller calls PerformActions after I report already made actions in ULearningAgentsController::EvaluateAgentController_Implementation. I looked into the code starting from PerformActions and I can’t really tell if it does anything significant for recording and/or training process so I assume the PerformActions doesn’t have to be called, but I’m not 100% sure.

THE QUESTION:
So what can/should I do in this situation? I mean I can think of 2 options:

1. Add a custom RunControllerWithoutSamplingActions function in my ULearningAgentsController child class that calls only
Interactor->GatherObservations();
EvaluateController();
without a Interactor->PerformActions call

2. I could add a boolean “bImitationRecordingMode” to my Interactor and set it in my IL Recording Manager class and in interactors PerformActions_Implementation just do an early return if it’s in IL recording mode

But maybe I just don’t understand something about IL training process and calling PerformActions is actually essential here and actions MUST be sampled even if the agent is an imitator character controlled by a human player?

4 posts - 2 participants