Sounds like your collider is moving in local space when you* should* be moving the pawn actor in world space instead.