Alignment Stuck CPU 0%

[WillBarley]

Hey guys, I need some help figuring this out.

I’ve been trying to align a set of images. Over and over it gets stuck in Alignment where the CPU falls off, progress message doesn’t change.
It’s usually not during feature detection, it’s usually during “processing”.
The only way to recover is to kill the program via Task Manager.

What’s happing to cause this?
I hoped that if my alignment settings are too difficult, alignment would fail or align few images instead of just sitting there.

Steps I took:
import 750 images
import masks for them
import flight log that contains position and orientation
group images by the phsyical camera they were shot with (3 identical cameras used) into 3 calibration groups and 3 lens distortion groups.

check the prior pose by doing
select all images
Selected Inputs > Prior Pose > Absolute Pose > Locked
then Align
Afterward reset to
Selected Inputs > Prior Pose > Absolute Pose > Position and Orientation

Then move on to real alignment
I have 3 cameras on a rig taking pictures and I have pretty precise location and orientation applied to them
so I am trying to figure out the alignment settings that work best
Here are the settings that have given me the best results - but 95% of the time i’m getting stuck instead of alignment.

explanation
postion accuracy is 1 meaning we are quite sure it’s within 1 meter of it’s prior position
orientation acurracy is 10 meaning within 10 degrees
hardness values we have tried to mess with, but they seem non deterministic
Detector sensitivity is Ultra to try to get features in low detail areas. the image quality seems very low noise where this might work

What’s going on?
RC version 1.4.1
latest GPU drivers for Nvidia T4 GPU (4 of them) on VM on Google Cloud

[WillBarley]

Tried some more things.

I’ve reset the app to Clean install many times, cleared cache many times.
Latest attempt:

start with project where the images, masks, flight log are imported and I have aligned the locked pose component to visualize the priors. ok then:

• reset app to clean install
• enable only ~350 images
• align with default settings
• change image overlap to LOW and align
• change merge geo referenced components to YES and align
• change add reconstruction region to NO and distortion model to DIVISION and align
• change detector sensitivity to HIGH and align
• enable all images (~750) and align → FAIL (stuck)
• disable half the images and align → FAIL (stuck)

[OndrejTrhan (otrhan)]

Hi Will,
your workflow is a little bit strange. Why are you providing the second alignment after the first one?
Also, how sure are you with the image’s coordinates, as you are setting them to Locked? Locked option is for a very precise locations in submillimetre precision. In your case I would prefer just Position and orientation setting.
What kind of camera have you used? How was the flight log created? Are you using local coordinate system?
If you are sure with the cameras positions, you should set higher Camera prior hardness, not lower.

[WillBarley]

Hi @OndrejTrhan

your workflow is a little bit strange. Why are you providing the second alignment after the first one?

first alignment (locked pose): Here I want to visualize the prior pose, the position and orientation according to the flight log only. This also allows me to select a group of cameras according to their spatial arrangement.

second alignment: Here I want a real alignment

Also, how sure are you with the image’s coordinates, as you are setting them to Locked? Locked option is for a very precise locations in submillimetre precision. In your case I would prefer just Position and orientation setting.

I only lock it for the first alignment - the one that shows me the prior pose. It’s basically a visualization of the flight log. For the second (real) alignment I set
Selected Inputs > Prior Pose > Absolute Pose > Position and Orientation

What kind of camera have you used?

This is a camera on a PCB - a part for integration into a robot, I don’t have the model number and I don’t have calibration for the camera. I have 3 of these cameras in a rig, capturing images.

How was the flight log created?

To create the flight log we took recordings of from an IMU alongside timestamps of the photos and interpolated the positions. We also provided orientations in the flight log because those are close to fixed. The rig of 3 cameras is fixed to the surface of the object I am scanning so we know the orientations with decent accuracy.

Are you using local coordinate system?

we are using Euclidian

If you are sure with the cameras positions, you should set higher Camera prior hardness, not lower.

What are the min and max values for hardness?
Should I also set the x/y/z accuracy values? What should I set them to? If I think that there’s a 90% chance that the position is correct within 1 meter, should I set the position accuracy values to 1? What if I’m 50% certain?

Most importantly, why is Reality Capture getting stuck?
If my alignment settings are bad, I am hoping for a bad alignment / too few images aligned.
But instead I’m getting stuck with 0% CPU usage and I don’t know if it is caused by my alignment settings.

[OndrejTrhan (otrhan)]

Hi Will,
basically you will get very similar (maybe the same) alignment visualisation using the settings set to Position and orientation (as the same coordinates are used there). As I wrote, Locked option is for submilimeters precisions.

Minimal value for hardness is 0, there is no maximal one.

Just set the accuracy if you know it. In that case it could be set to 1 meter. If 50% certain, then I would use 2 m.

In the processing stage of the alignment are used CPU, RAM and Storage only. So maybe it can be the problem also there.
Your settings seem to be normal, so maybe this will be the issue with used coordinates. Hard to say why exactly it is stuck. Have you tried the alignment also on the physical machine?
When you start the process without using the locked position and go right away to Position and orientation, are you able to align the data?

[WillBarley]

@OndrejTrhan

basically you will get very similar (maybe the same) alignment visualisation using the settings set to Position and orientation (as the same coordinates are used there). As I wrote, Locked option is for submilimeters precisions.

I am not finding that to be the case.
If I do a real alignment, letting Reality Capture change the positions, I am getting a lot of residuals. Example:

If I want to see the prior positions before Reality Capture messes with them, I can lock pose and Align. This also computes much faster than a real alignement.
This would be a good feature to add in a full fledged way instead of how I am doing it.
Do you think doing this has a negative effect on the real alignments I am doing?

In the processing stage of the alignment are used CPU, RAM and Storage only. So maybe it can be the problem also there.

When it seems stuck i’m not seeing any CPU, RAM, or disk usage.

Have you tried the alignment also on the physical machine?

I have not

When you start the process without using the locked position and go right away to Position and orientation, are you able to align the data?

I will try this. Is your theory that each alignment affects the next? And that my Locked alignment has a negative impact on further alignments?

[OndrejTrhan (otrhan)]

Getting results like that could mean, that something is inappropriate. It could be the camera parameters, computed positions or rotations or precisions. What is the resolution of the used images?
Aligning using the locked pose is quicker as the app is using these information as determined, so it doesn’t need to compute their positions. But if the parameters are not precise, you won’t get the proper sparse point cloud and then model (your sparse point cloud is quite sparse using the locked positions. For ordinary cases it should be more dense).
Yes, each new alignment in the project takes something from previous alignments. So, each new one could be more precise. Or, if it was wrong at the beginning, it could be worse…

Would it be possible to share your data with us? If so, I will send you the invitation for the data upload.