I’ve noticed similar issues when I put all of my photos in at one time - for me it was specifically a perpendicular intersection in what was a flight plane (some cameras were thought to be perpendicular to what they really were). What I’ve done is organize the photos into subgroups, (low orbit 1, high orbit 2, etc) and try to keep the # of photos/ group to be ~50 for the first few. I then add one, align and look at it. As the ‘skeleton’ gets built, the groups that were more details or just had more photos ended up being added fine.
Might be worth a shot.