How to reduce penumbra shadow noise (Virtual Shadows & lumen)

Hello, I am currently trying to figure out how to reduce penumbra noise for directional light, if you have a source angle set to 5 (Using Virtual Shadow Map & lumen software raytracing). I already figured, that the supersampling has some impact how much noise there is. Using DLSS instead of TSR gives better results, but has still some super heavy noise if you move around in occluded shadows (see picture). Theres also a section about controlling shadow penumbra quality in unreal docs, but increasing the SMRT Ray Count, did actually not help at all for me.

So - does anybody has some experience he could share, how to get rid of this noisy penumbra/shadow?