I would still use possess for the driver. But other than that you’re right. Not the best solution.
What I would do for passengers:
When the player hits the interact button and enters the car as a passenger I would attach the player to the car. You can use sockets for positioning within the car.
For the input: I use Enhanced Input in my project. I would add a new input mapping context. Within this context only basic input is available like looking around, aiming, shooting and interact to exit the car. No movement, jumping or anything else.
In that case you don’t have to do any checks within your movement functions if he’s in a car or not.
When he exits the car just switch the mapping context back and you’re good to go.