If an engine dev or expert could give advice here that would be super helpful to both of us.
I’m working on the exact same problem and I have to say, it’s pretty difficult. I’m pretty sure I’ve figured out all the things I need to do to make the lighting work properly, but it’s a lot of learning. I found these major problems with lighting in instanced levels.
Rotation of lightmap SH:
This isn’t super easy to fix, but I can see two potential solutions. One is to change lightmass to bake SH in the meshes local space rotation instead of global, and rotate the SH sample vector in the shader since we usually always have the local space uniform available (right?). I don’t know where to implement the change in lightmass though. Other solution is to accept lower quality lightmaps and reduce the 2nd order SH to a 1st order SH which doesn’t depend on rotation. I think UE4 already has low quality lightmaps like this as an option but I don’t know exactly how to make it work.
Indirect lighting cache transform and resulting sampled SH rotation:
This one isn’t so bad I don’t think. If I add a transform matrix member to the Indirect lighting cache object I can transform any sample locations into the “lighting cache space” and sample that location, then inverse rotate the sampled SH coefficients back into global space. Don’t have to change anything in the shaders for this but it does add a bit more CPU load. The upside is that the level transform won’t change so we can precompute the 2nd order SH rotation matrix to speed up sampling. I could instead maybe duplicate the actual lighting cache data for each instance and pre-rotate all the SH, trading memory for speed. Since I happen to only need 90 degree rotations on Z, I’m hoping that it will be easier to implement the actual math though I am still way out of my depth here.
Reflection probe rotation:
This one seems rather simple for rectangular probes, it’s a one line change to the shaders sample function which rotates the sample vector into the probes local space, making the reflection environment rotate with the actual reflection probe component. For spherical probes it’s a bit harder because I need the component rotation in order to transform the sample vector. Problem is I don’t know how or where to retrieve that value and pass it as a uniform into the shaders.
Also you’ll find that BSP meshes don’t work properly, but that’s not a big deal and it’s easy to fix (see how the world composition streaming does it). I haven’t even researched how this effects navmeshes yet but I’m sure it’s going to be full of problems as well.
So I’m working on a solution that I might be able to share with you, but I still need guidance from an engine dev or some other expert on the best course of action here.