I just accidentally started colorizing the wrong component (a very large one), so I hit abort right at the beginning. Then something strange happened – all of my points for that component disappeared.
I thought maybe the points would come back if I colorized it again, so I tried that. Instead, RealityCapture switched my component component selection to another one and colorized that instead!
When I saved and re-opened, the large component was still missing all its sparse points, but now the “colorize” button was greyed out. I re-ran the alignment and the points came back and all was well.
I tried to reproduce this and cannot. So I figure it must be some weird edge case and I’m just reporting it in case it happens to anyone else.