About Learning Agents

[Deathcalibur (Brendan Mulcahy)]

Your definitions for the episode, steps, and iteration look good.

To be clear, an iteration is triggered to train when either the max_episodes or max_steps thresholds are crossed, whichever comes first. Most likely you would be crossing the max_steps if using the default settings.

@masaokun_lover

I have a few more questions,
The average reward in the log is output every 78 iterations, but how is the average reward, average value, etc. calculated (please also tell me how the average reward is calculated when the log is output)?
Also, could you please elaborate on the relationship between episodes, iterations, and steps?

The log is outputted every training iteration, not every 78. The averages are computed by using the mean(). See {Your-Unreal-Install-Dir}\Engine\Plugins\Experimental\LearningAgents\Content\Python\train_common.py, line 323.

@masaokun_lover

Key questions we would like you to answer
What is the relationship between the number of episodes, iterations, and steps?
How do you calculate the average reward for log output?

I think this is explained above. You define the settings for the steps/episode threshold in the training settings. I believe the default is 1000 episodes or 10000 steps. Whichever is crossed first will trigger a training iteration.

@masaokun_lover

We need to include this information in our papers, etc. … Thank you in advance for your time and help in answering this question.
(Again, I used translation software to ask this question.)

No problem, happy to take a look at any relevant paper that is using Learning Agents (assuming you have an English translation available).

Good luck!