Okay so i did a test like this:
Level 1 has several hundred static mesh actors (nanite enabled). Each static mesh actor is composed of a handful of high poly columns. The total amount of columns is ~3,500.
Stats read (i will round to nearest since there is small fluctuations each frame):
120 fps
total frame time 8.3 ms
GPU: 4.3 ms
Draws 240
Prims 320k
Level 2 has same number of columns, same materials, but there are several hundred blueprint actors, each containing 12 ISM of that same column.
The stat readout is pretty much identical.
I am not sure if there are any other relevant stats to look at. I also wonder if there is anything important missing since this only has one static mesh. Because if we have a good amount of unique static meshes, thats where a difference would most likely be seen, as compared to if all the buildings are just composed from a handful of static meshes that only needs to load once.
But the instancing workflow is so much slower and more complicated, I really hope to avoid it if possible.
As @RecourseDesign suggested I ought to test with a lot more materials at play as well. But further test will take more time to setup.