I ended up reducing the blend factor to 0.5f, as it seems to reduce jitter on very noise stuff. Temporal AA sharpening - Rendering - Epic Developer Community Forums
There might be some stuff that is more visible by reducing this, but in my scene it looks rather nice.
BlendFinal = clamp( BlendFinal + sqrt(sqrt(distance(BackN.xy, UV.xy))) * 0.5f , BlendFinal , 1.0f );
Here is from a moving camera.