Ray traced distancefield, meshes shadows flicker

Well unfortunetly the flickering continues no matter the resolution of distance fields.
I increased it to 3 and then 4 times , and in the vizualizer it looked axactly like the mesh, very good and solid. problem is it still flickers with distance set too high.
I found that some other objects in that same area also flicker in the vizualizer, they are just covered by other shadows and I didn’t notice them. Like some very small meshes - small pillars . In the video below you could see that.

I also tried using a replacement mesh for the distance fileds of the tower, with their exact size but just solid primitives . Their DF looks as expected , but shadows still flicker.

More info on the tower that might be relevant - its made out of 3 big pieces , there are big windows on each floor . The windows are part of the mesh and are made out of two elements - outside element with Masked material set to it, and inside element with transparent material . I did that because of an issue with fully transparent windows, rendering one over the other when looking from outside. I don’t know if that might be an issue.

Also the small object you can see flickering in the video are simple meshes and have pretty good looking DF .
video with DF viz