You can, your logic is wrong, see my answer above.
Lets say you have a Wrap Widget with 3 children.
You are currently looping through 3 items and removing the item at that index.
Loop iteration 1:
Remove object at index 0 (2 items remain)
Loop iteration 2:
Remove object at index 1 (1 item remains)
Loop iteration 3:
Remove object at index 2 = Error: Accessed none trying get hold of the 3rd child of the wrap box, because only 1 remains.
Remove your link from the index of the loop to the child index, so you are removing index 0 every time, and you should be fine.