I managed to find the solution 
I went to project settings > physics and then changed the Bounce Threshold Velocity to a low number. This allows objects to bounce even if they are traveling slowly, rather than just stick to the wall
I wouldn’t recommend setting this too low or to 0, because I imagine it could cause some lag if too many objects are able to bounce continuously.
