Pawn starts falling on client after not moving for a while

I have a spaceship pawn the player can possess and fly around with, driven by this logic:

Since this is supposed to work in multiplayer, I let the dedicated server do the actual changes and let client interpolation take care of the rest. I also set linear and angular damping to 1 to automatically slow the ship down, when no input is given. However, when coming close to a standstill, the pawn just falls down as if there was gravity (note: its flying in space, so there is none). I figured, it has something to do with client interpolation, since when i press any input, the spaceship snaps back to its old position, before falling down, but doesnt it remain there in the first place?