VSM serrated artifacts

Using Virtual Shadow Maps for rendering, I keep on getting these ‘serrated’ shadows - like the system doesn’t understand what resolution to render at.

I don’t manage to get rid of these (UE5.1.1). Is this a bug? Can it be caused by the setup that I’m using? Using Ray Traced shadows, the issue disappears. Disabling Nanite also solves the issue. Obviously, both those ‘solutions’ introduce other problems.

What can I try to resolve / work around this?