Cascaded shadows lose quality when increasing draw distance

This is the situation I face with realtime shadows.

It looks like the reason is incorrect work of the cascaded shadow maps.
For an unknown reason the bigger the cascaded shadow map becomes, the worse quality the shadows have. As I increase draw distance, the shadow’s offsets increase and they move down from the origin (same effect as if I increase the bias), so at certain point previous cascade almost completely disappear under geometry.

Contact shadows can be a partial solution for this situation, but they have lower quality and, AFAIK, might be too expensive to use it everywhere. I assume that increasing the overall scale drastically can also help, but I’m almost sure that gigantism will generate it’s own problems. Is it possible to somehow fix the shadow’s runaway?

That is totally expected behavior.