Specifically I think its soft shadow noise you’re seeing. Both Virtual Shadow Maps and Raytracing have noise in their soft shadows due to how they sample a limited number of rays per pixel - to test this set your sun’s source angle to 0. This will make all shadows hard (which we don’t really want - but just to test) and I’d be willing to bet all noise will be gone.
As @SoulReapX mentioned, Increasing the RT light sources samples per pixel will help. TAA/TSR also usually helps shadows look more stable by averaging noise out over multiple frames.
Cascading Shadow maps do not have this type of noise but virtual shadow maps soft shadowing “SMRT” feature do - even worse than RT - which can also be reduced by increasing the ray and/or sample count vars.
Even going from 1 ray to 2 rays will massively reduce noise in RT but returns are diminishing as you go up for RT or VSMRT. You’ll probably reach an unusable frame rate before removing all traces of noise, just one of the limitations of real time ray tracing as far as I’m aware.
If this is a critical problem, you could try to implement an alternative denoising solution such as Nvidia NRD. Its already in the UE4 RTX branch, presumably it could be ported to UE5 but that’s out of my wheelhouse.