how to make your pawn "slippery" and bounce of walls while falling?

Hey everyone

so kind of a weird question here. Basically I have a game with really high towers and if you fall through an invisible collider it triggers a “death infinite fall”. Problem is, sometimes my towers have small colliders that the player can land on or climb up. So it looks weird having your character landing on a small collider of a vertical wall of the tower and then have a game over.

What I would like is to have the view of the pawn just falling down unable to grab or land at anything, and would bounce of walls in that state or roll over small colliders.

What would be the way to achieve that effect?