Keeping track of attached actors - blocks on blocks on blocks

Scenario - the player can pick up blocks(actors) that are simulating physics and attach them to other blocks that are simulating physics and create structures from them, that also simulate physics, my question is how is the best way to handle the connections? I have experimented a lot, making the first block be the parent for all attached blocks so in the hierarchy you have 1 block with all the other connected blocks attached to it i.e. Block>(Block1,Block2,Block3). I have also have tried attaching each block directly to the block it is connected to, i.e. Block>Block1>Block2>Block3 etc.

I want to be able to ‘detach’ a block and have the other blocks in the structure remain attached if there is another connection somewhere, or detach if there isn’t. The problem with the first method is if you have a chain of 3 blocks and you take out the middle, the 3rd block is still attached to the 1st, to fix that I have tried making an array of connections as a variable in each block and checking the array to see if it is connected directly to the parent block, but if you get many blocks in a chain or loops going, then I have to loop through arrays of arrays of arrays which I can’t seem to figure out how to do logically to check if there is a connection somewhere to the parent block.

I could go on with things I have tried, I have searched quite a bit on the net for solutions, maybe I don’t know the right words to search, but I’m hoping someone out there understands what I am trying to do and knows of an established method/algorithm out there for handling this kind of thing. All suggestions appreciated.

Seems to me the simplest method would be using physics constraints between each block, but this doesn’t work because they are so costly and aren’t as rigid in a chain as attaching actors to actors.