Solved it! As I was debugging various stuff in my game, I noticed the player character was somehow attached to the car in the World Outliner. That allowed the player to move independently from the car but was bound to any move the car was making.
And I was also wondering why the player mesh was following the car when driving it. I detached the player from the car and all is fine now.