Weird blurry foggy noise on plane surface when far away from texture

I have a surface that needs to be viewed from very far away. Before building the lighting, it looks like this, which is fine:


After building the lighting, it looks like this:

For reference, I am looking at the surface from about 500 meters in the sky. The effect also is reduced if I put certain different textures on the surface using the same material. There is no Atmospheric fog in my level.
How can I build lighting without getting this noisy fog?