If it was me and i wanted something quick id probably put in a state that says your on a Ladder and then disable any inputs you dont want to affect the character while hes on a ladder.
So if im on a ladder i can only move up or down, not on any other axis as those inputs would be disabled while on the ladder.