Ok, so got some time at the PC. Here is en example of the issue:
First image with sun spot on plane. This is what we want consistently when moving the camera:
How it looks with Visualize | Lumen Scene:
Now with r.Lumen.DirectLighting.ForceOffscreenShadowing 1 
Notice the floor is completley black here.
Result when looking up enough for sun spot on plane to go off screen:
Notice how room got darker and all bounce light from floor is gone.
Now fixing the issue, this is how the floor should look in Visualize | Lumen Scene with the console command active:
Now this is the result when looking up:
Now Lumen preserved the light bounces from the floor even when the Sun spot is off screen.
Video of me breaking and fixing the floor for Lumen: https://www.youtube.com/watch?v=pYIpFcahGo8
Hope this helps.