One last random thought, probably blog this elsewhere, in fact I’ll do that now! For each cubemap, store albedo depth and normal (compacted to two channels for normal and albedo, quality not that important) then with originating camera position you’ve got all you need to relight say, one 256x256 cubemap face each frame, resulting in however many (in this case 5) newly lit cubemaps a second.
Keep the the relighting to say, the sun/primary light and whatever changes to the skybox, and you’ve got a relatively cheap dynamically relightable image based lighting system, for relatively little coding effort. It shouldn’t even be that GPU intensive, you’re just re-shading a fixed screenbuffer with a shadow map applied, no draw calls or anything else needed. Heck do 3 faces a frame and you could do 150 cubemaps over ten seconds, assuming your primary environment lighting doesn’t change that quickly it would all be fine.