Let us know your PC specs. I tested several different models, adjusting their positions in the scene and changing the camera view (from any angle), but I couldn’t replicate such a large performance difference just by displacing the mesh. In fact, I barely noticed any change, even with a lot more meshes than in your scene. So, I think the issue is likely something else here, as it might be a compatibility or driver issue (or a bug in 5.5 preview). You can share the project with us so we can compare the performance on different systems.
I don’t believe either the idea that “high poly makes Nanite faster” but it might be true in some cases depending on other objects that occlude in a complex scene. Similarly, when someone said “More LODs can reduce object popping and improve performance” it’s true to an extent, but using 30 LODs per model could actually make things worse.