Blueprint and LOD's

Hi, I have a blueprint of a building that is procedural generated using instanced static mesh to build a city quicky. At the moment I am doing the LOD distances and setting them up for performance to use with VR. At the moment what is happening is, I have LOD 0 pop’s in at 1 meter (1.0) and LOD 1 outside 1 meter (0.98). performance is good. But the problem I have come across is, because it is a building and from the HMD you maybe looking at the front of the building and the LOD 1 goes to LOD 0. but when that mesh pop’s in, it is also popping in the mesh on the other side of the building (other walls) that is outside the 1 meter LOD distance. what would be the best way to stop this from happening.