I’ll just throw this in here. Not sure if it’s relevant. While I never had a severe drop in performance, one thing I did struggle with was that my shadows would flicker. Basically, my capture camera was in a far away land with its own surroundings. This caused the lighting near the main camera to switch to distance field shadows, but it couldn’t keep up. So the main camera would display bad shadows. The two cameras were fighting for what shadows to use and the engine couldn’t keep up when the switch happened twice per frame causing flicker.
I moved the capture camera closer to the main camera and everything was fine.
I’m not using Nanite, but I’m wondering if Nanite isn’t more sensitive to camera location changes.
I’d be curious what happens if you just put your capture camera NEAR the main camera and also aim it in the same direction, but only render what you have set up for the capture (2 cubes or whatever). This would ensure nanite visibility shouldn’t change, lighting shouldn’t change (though you may need to keep the same lights on/off), shadows wouldn’t change, etc.
Just a thought… possibly futile, but thought I’d mention it.