How to prevent switching LODs on camera rotation?

How can I fix the problem that when rotating the camera (or on VR, rotating the head) the screen size of meshes changes and LODs are switching?

Either the screen size needs to stay constant when rotating the camera (rotating the head), or the LODs need to be based on distance in world units and not on screen size, or is there any other solution to this?

It looks very ugly in VR to constantly have LODs switching on the edge of the view. I can’t just let the LOD transition happen further away since performance is so important in VR.
Having a very visible transition between LODs is completely fine as long as it only happens when the player moves, but constantly have everything flicker around while looking around (rotating head=camera) obviously looks very bad.

Im not sure if its only screen space that is changing…

When I am standing very close to a trigger volume (in the FPS template for example), a simple rotation of the camera will trigger an overlap event.
For example, standing next to a BP trigger volume activated door, simply looking a bit left or right, opens the door without the player actually changing position.

As if the collision volume of the player were not really symmetric…