It is my opinion that you should not be building your buildings wall-by-wall like you are in the engine. Each of those meshes is an extra draw and you’re trying to optimize whatever you can for virtual reality. Combine all the walls inside of your 3D modeling program to have one complete building. To get better framerate, you should also get those lightmaps made and have static lighting.
I also don’t think you need each of those bricks to be separated like that. Those buildings don’t need to be anywhere near the 100s of thousands of tris you are putting into them. On top of that, those bricks need to be beveled and have awesome textures to get the feel you are going for. So I would recommend relying on textures to give depth to the brick, and have a few bricks here and there sticking out with geometry. It’s okay to have optimized, mostly flat buildings. With a good enough environment and detail, it won’t really matter that they are a bit flat in VR.