As a starting point, I would try adjusting the manager tickrate from 0 to 0.2 (5 Hz) and then adjust the cubes movement until it works well. Your interactor’s actions are an issue because the agent will only move on the frames where the interactor is ticking. I would have the interactor output a velocity and then have the cube apply the velocity every frame in its own tick, e.g. create a public SetVelocity function on the cube and then have the interactor call it periodically.
The reason I want you to make these changes is because it can be hard to learn to control an object when ticking at a very high rate.
BTW for your issues with gyms, we’ve added a simple gym manager called ALearningAgentsGymsManager and a gym class ALearningAgentsGymSimple, which make it easier to spawn and manage basic gyms like you need. I don’t recall however if this was added by 5.5 or not, and also its an undocumented feature so you’ll need to look at the two classes I mentioned plus ALearningAgentsGymBase to figure it out.