I’ve had this a few times but narrowed it down to:
[list=][*]Too few images with CP in current chunk
[*]Misaligned image CP
[*]Distortion parameters not optimal (CP near edge of image)[/list]
The latter was mostly with fisheye images where the very outer edges of the image should be avoided for CPs but may be possible with “normal” lenses. Checking the tie points on the images with the CP may provide some clues if there are alignment issues throwing off CP locations