Keeping this collision would be the equivalent of using Complex Collision, pretty much.
Double click the mesh, Remove Collision from the Collision dropdown at the top bar. This will remove the collision approximation (a.k.a. Simplified Collision).
In the details panel, set Collision Complexity to Use Complex Collision as Simple
See how that behaves. You can visualise it here:
