Thanks for reply. The mesh is not that small or low poly, I reduced the face count and complexity to a box to figure out what is happening.
It seems when enabling nanite on a collision mesh, it tries to simplify it.
You can try this by importing same mesh from FPX twice, one with nanite enabled and one without, then use each mesh as collision for other one and see the difference (show collisions in mesh details window).