How to keep a reference to a fractured destructible mesh?

Hello, I’m struggling to wrap my head around destructibles so I’m here to seek some advice…
I have managed to setup a my player destructible pawn class, its an asteroid with physics enabled and its reacting to collisions with other asteroids. So far so good…

I can fly it like a space ship and everything works until it gets fractured, in fact when there’s enough impact damage to fracture the mesh the pawn “looses” the asteroid, what I mean is that the reference to the asteroid seems to break and I cannot fly it anymore, while it just flies away in pieces, why is that?
How can I keep it attached to the class?

The desired behaviour is like: I’m driving a car/plane, I hit something and part of my vehicle breaks apart but I still can drive it.

Maybe I’m doing the wrong thing? If so how can I make the player character destructible and keep control on it?
I’ve seen people talking about pre-made fractures in blender or houdini
But I wonder why you should fracture things outside the editor if there’s a physics plugin for it?