I have a similar problem: the collision disappears after deleting a shard.
If I split the plane into one level (without clusters) and set the lifetime (removal-on-break) to zero or slightly higher, then after the Geometry shards disappear, the collection stops responding to the check and the collision disappears. It’s as if the actor has disappeared from the world, but continues to render.