Update:
After trying new reward scales, adding overhead obstacles, and getting rid of pitfalls entirely (as well as removing the fall off penalty) and then training for a few hours the agents are still jumping any chance they get. I also added a jump penalty (negative reward on condition for if the agents’ z location is 10 or more above their original z location) but no change so far even after resetting all 4 networks and retraining.
After some more looking I still can’t find how to adjust action noise unfortunately.
I’m gonna keep trying new fixes and post a new update later today.