Are your rooms basically inverted-normal cubes with paper thin walls imported as a single mesh?
If so, that’s completely against the way epic suggests you build environments in the engine. Each wall should ideally be a separate mesh that’s water tight and as thick as a wall really would be. Yes this means that you don’t see a bunch of the sides but that’s not going to affect performance in the slightest. On top of that, you’ll get better mesh distance fields, better occlusion and better frustrum culling. It’s also a better work flow since you don’t need to reimport the entire room if you just want to change one wall in the editor. Oh, and it will be way better when it comes to making collision.
You will always get better results if you work with the tech rather than in spite of it.