Group actors into groups and remove them en masse

You’re better of just setting up proper culling. Another option would be to use prefabs. This will make your destroy list a lot smaller. The list itself would be TArray (Actor Object Reference).