It appears that not instantly un-possessing the pawn makes the physics actor stay in the right location for the owning client and the server.
This doesnt help with other networked clients though.
Is anyone else finding issues like this? I cant find anything in the patch notes that would hint to how to fix this.