Honestly? That looks like essentially the correct behavior for lumen. Lumen does diffuse GI very well, but mirror reflections aren’t going to look good if there isn’t an analytical light source to let lumen do the NxL on. If you increase the roughness by even a bit, a lot of that effect will go away.
The GI is screen-dependant because the surface cache that lumen traces against does not (and without hit lighting, cannot) match the GBuffer lighting. The best you can do is hide screen traces, or disable them, but you can’t create a good quality match in the worst-case scenes.
Plus, unlike standalone RT reflections, we have accurate specular occlusion, but I still hear your frustration with some of lumen’s limitations.