Okay the Denoiser does actually seem to be the main problem, I’ve tried your approach and it did fix my shadows, actually the only thing I had to do was just disable the Denoiser. Easy workaround to get your Ray Traced shadows back.
Now obviously depending on your lights there will be noise in the shadows, though at least in my scenes it was barely noticable.