So as an update to this situation, I went through the light settings, the mesh lighting settings and none of them changed any of this. So I went into the Mesh>LOD0>Build settings and found that Two-Sided Distance Field Generation set to ON resolves the issue above. You then no longer need to have the light on screen to have the bounce lighting work as expected. Not sure why this change makes it work better, the performance difference seems to be unnoticeable and since UE5 isn’t finished yet, there is no description for what this option is supposed to do when you mouse over it. I just tried every setting until I found this one and it worked.
If anybody knows why it’s working (on a technical level), please let me know.