If you’re only climbing to fixed ledges, is probably best to use root motion as GRIM_Warlock said.
Motion Warping might be useful here to smooth it out.
If you’re going for some sort of freeform climbing or ladders with an arbitrary height, you might want to try setting movement mode to flying (or a custom mode with no gravity), then use add movement input instead off offset loc, and let the movement component handle the interpolation.