Hi,
did you tried to reduce the Push Force Factor or disable Physics Interaction to test in the Character Movement Component of the Pawn?
Optional:
You have to edit the Physics Mass from the Physics Asset of the Vehicle(Mesh) to have effect on the Push force from the Pawn.
Also dont forget to go to your Vehicle Movement Component and setup the Wheel Trace Collision Responses for Pawn to ignore if you got the problem that you still push the Vehicle.
Thats how it worked out for me 