Not sure, it should not jitter a lot with rewinding a lot especially in single player mode, there could be something wrong with the tick order that time. Sometimes if you are forcing transforms from GC to SKM or vice versa Skeletal mesh copying GC transform in wrong tick groups can cause that aswell, it will cause major desync.
On damaging side , in car case since its almost a single vehicle its possible to create layers of geometry collection.
For tires it can be 2 bones 1 geometry collection and for body inner chasis can be single bone, outer parts can be fractured many times. Root bone and inner chasis can have higher damage for breaking something very very high.
Upon collision or tire part removal it would still go, and it will effects its physics as it should be however think density of these parts can be lower to uneffect the overall stability of the vehicle.
In my pod its the same think can do something like that however it would require 3D modications anyway, think on your case too properly assemble the vehicle which is fine can be done, its just a bit time consuming.
Here below is some of my pure GC vehicles in multiplayer. With some custom arm components.


For me with the pods, its just about the rewind causing problems with niagara (for now) and sometimes with gc sockets (but there is workarounds for it, registering non simulated gc bodies with cluster solves it). Some devs feed velocity to niagara and compansate however its not super reliable especially vehicles with high velocity like a pod racer.
If we can mostly get rid of skeletal meshes its possible to do more accuracy with chaos ofcouse which will avoid frame syncing between game thread and last phsyics async. Since this brings the question you ask “What about fracture?”
When GC is the truth on async and if vehicle hits the hidden geometry collection either hit earlier or delayed. Since our eyes follow the skeletal mesh which is anyway delayed there would be some mismatch in fracture timing and placements. Can be ok in low speed however in high speeds would be visible.
Below is the difference on SKM Thruster and GC - Turquise is the skeletal mesh, difference is acceptable for now.