Ok that’s good to know it is a timeout. That starts to get us somewhere 
Where should I change this value? I’ve been looking for this parameter for a while because I found that the agents were always reset by 300 steps but not the steps I defined in the
Event Set Completions.
This is in the Trainer Settings which is a struct you pass into the Setup Trainer node during BeginPlay. But I’m guessing this isn’t the issue if you haven’t touched it. I was concerned that perhaps you had increased it significantly.
So the way that training data is sent to the python process is that the gathered experience is put into a buffer, and when that buffer is full, that triggers the training iteration. The “fullness” is controlled by the MaxEpisodeNum or the MaxStepNum on the Trainer Settings:
If you’re using the default settings, you’re typically going to be hitting the Max Recorded Steps Per Iteration once you have ~33 episodes (10000 / 300, assuming you’re always hitting the max step in each episode). I think you’re hitting a timeout from this taking too long, although I don’t remember off the top of my head. By adding more agents, you’re significantly speeding up the time it takes to gather those episodes.
Let me know if you run into further issues but it sounds like you’re back on track?
Brendan