Nice looking scene!
You were definitely on the right track, but the wrong shadow type.
Try adjusting the cascaded shadow maps and set it to higher value.
