The indirect light disappears because VXGI has limited range of its effect. You can see that range by switching the view mode to either VXGI Opacity Voxels or VXGI Emittance Voxels. The range can be increased using the r.VXGI.VoxelSize and r.VXGI.MapSizeX/Y/Z variables, but for this scene a better solution is to place a VXGI Anchor actor somewhere near the building.