I’ve been having a similar issue. Was trying to tow a vehicle and it worked fine when playing as Standalone or ListenServer, but when playing as client it would only work when the ListenServer had the vehicle in view. I experimented a bit further and even just pushing a vehicle in the UE5 Vehicle example project as client was causing the same issues you were experiencing.
Looking through the code I found that for some reason UChaosVehicleMovementComponent sets
SkeletalMesh->VisibilityBasedAnimTickOption = EVisibilityBasedAnimTickOption::OnlyTickPoseWhenRendered;
in UChaosVehicleMovementComponent::OnCreatePhysicsState(). This seems to override any setting on the SkletalMesh properties.
So to solve the issue I set
GetMesh()->VisibilityBasedAnimTickOption = EVisibilityBasedAnimTickOption::AlwaysTickPoseAndRefreshBones;
in BeginPlay() of my vehicle class.