I have followed the tutorial on reinforcement learning and have it working decently in my game in a simple use case, but for more complex use cases I have having issues with the AI learning very slowly, or getting stuck in a local maximum.
In some of these cases, I think starting with some initial imitation learning would drastically accelerate the process and I have behavior tree based AIs that can execute portions of the behavior I am trying to have my learning agents do. I would like to use the behavior tree based AIs (or potentially a player controlled pawn) to record data for imitiation learning.
I am trying to set up the recording like this (here the agent being added is a pawn controlled by a behavior tree):
then once my actors are all initialized and ready for training I am doing this
However, I am not clear on what the learning manager class on tick should look like.
I can do this
If I print the output of Get Observation Vector everything looks good and it appears to be correctly gathering observations from the agent that is controlled by a behavior tree.
I then added code to call “setFloatAction” for each of the interactors actions and used game data to calculate a float action that is equivalent to what the behavior tree decided to do (such that if a learning agent called getActions with the float actions set to those values then it would do exactly what the behavior tree controlled agent did). I can also print those float actions values and they look proper and match what the behavior tree is doing.
I believe the next step should be to call Add Experience like this:
However, when I make that call I get console errors like this:
LogLearning: Warning: BP_FighterRecorder: Agent with id 0 has not made observations and taken actions so experience will not be recorded for it.
If I try to call “getActionVector” I get a related (I think) error
LogLearning: Error: BP_FighterInteractor: Agent with id 0 has not yet computed an action vector. Have you run EvaluatePolicy or EncodeActions?
However, as far as I can tell there is no EncodeActions function and I don’t think calling EvaluatePolicy makes sense (there is no policy, I am just trying to record…
Any help is greatly appreciated, thanks!!!