Create and apply a physical material to whatever is responsible for the collision of the rolling ball:
If you also want to prevent it from bouncing up and down, the easiest way to achieve that is to place a collider above the ball - make the Blocking box collision a part of the pit.