Normal - Lumen makes material flicker

When making a texture with Depth fade, I have this issue where it would change to the color of the fog instead of the color of the material, and the colors will sometimes flicker based on the camera POV, as shown:

This is without Lumen (working):


And this is with Lumen enabled:

Repro:
Make a foggy scene
Add a plane with a material with depth fade on opacity

EDIT: Seems that the plane is what is causing the issues, even with a normal material without depth fade it will do the same (test 2 was with a plane on opacity 0.7).

Test 3 was a plane without opacity control, just a color. I still get the same issue:

Test 4 is placing it on a cube rather a plane. Still the same:

I give up for now, hopefully, this report will help spot the issue. Material:

This is happening on other areas of my game, and its making it hard for me to use depthfade which is a very useful thing to have in large maps