Alignement problem : Colored scan with images

[Anonymous_90647c4b34]

Hi,

I have been experiencing an issue with the image align phase. I have captured several images using a DSLR camera equiped with a 15 mm fisheye lens, and of the same zone 20 scans colored with the internal camera using a P30, registered in Cyclone and exported as .e57.

For some reason, The software RC cannot match scans with images and created two separate component. Here is an example of the image from the scan and an image from the camera :

On previous projects where the alignement worked beautifully, I used for coloring the scan and for the photos the same camera, but with different lenses (a 24 mm for the photos, and a fisheye 8 mm or 15 mm for the scan position panorama used for scan colouring). Even for photos shot only on a small part of the scene, the photos were correctly aligned (which amazed me).

I also had some projects where the scans where colored with a fisheye lens (15 mm) at the same nodal point position (thus just a e57 exported from Cyclone), and many pictures taken both with a normal lens and a fisheye lens and it worked.

So my question is why does it fail when the color information for the scan is taken with the internal camera of the laser scanner ?

Also by extension, if we do not color a scan (using only the intensity) can RC match pictures to the scan (or the space used for calculation treeat the information fairly different) ?

Finally, is it possible to color 2 or 3 scans, make the rest in intensity only and use external photos for colouring ?

Kind regards,

F. Poux

[Anonymous_67437ca36e]

rc dosen’t really support fisheye lenses.

but you can have a go using division. I’ve had a few goes with a 10.5mm and some worked, and some crashed. but i wont really bother trying again until rc gets updated. it was a bit hit or miss, and generally introduced more issues than it was worth. esp mixing normal photos with fisheye.  

fisheye images with the scans should be fine if you colour them before they are imported to rc.

rc is matching by colour, so it really is needed. otherwise you could have a go with control points and see if that helps. 

 

[Anonymous_aa64e24c61]

Hi Florent,

did you make sure to export the e57 as ordered, so that RC can determine each scanner position? If it is only on file, then there can be severe problems which would explain the failing to align scan and images.

How did you take your images? It sounds like you are taking panorama shots from a few standpoints. Also, large parts of your object have less than ideal surfaces where RC probably has only very few tie points, if at all. You can check that in 2D view by activating Tie Points in the blue Image menu.

I’ve heard people claim that it also works with intensity only. My guess is that that depends on the object, since for some surfaces RGB and intensity are probably close enough for RC to identify some common tie points. In theory, you only need a handful of matches for a general alignment. I was going to try it out soon myself - if you get there first let us know how it goes! I would pay attention to small detail though because it could well be that there are larger deviations from the scan in areas where the texture does not allow RC to find matches.

[Anonymous_ac400dd84d]

Gotz, what do you mean by “… where RC probably has only very few tie points, if at all. You can check that in 2D view by activating Tie Points in the blue Image menu.”?

Sounds crucial but I can’t find anything that looks like a ‘blue Image menu’ or any ‘Activate tie points’ button!

[Anonymous_aa64e24c61]

Hi Tom,

At the top menu bar, there is a blue menu called “Image” (next to Reconstruction) which only appears when you activate a 2D window. There you can set what you want to have displayed - works accordingly with all other windows (except console I think).

[Anonymous_154bee8614]

Hi Chris, Götz, Tom, thanks for the answers.

@Chris > Ok for fisheye lenses. Indeed, looking at the components from the fisheye lenses, while matching steps get passed successfully, hard to get a viable and precise reconstruction out of it alone (thus wanting to use the 3D registered laser scan point cloud as a reference). Divsion gets worse results, which doesn’t surprise me looking at the mathematical model (I want to highlight that it is objective remarks, and that I am impressed by the speed and the quality of the reconstruction when using normal images !! Hard to find better on the market, and no competitor for merging laser scan and photos).

Ok for colour, but does it matter where the color comes from (internal camera or colouring through pano head with DSLR) ? Indeed, as you may know, there is a deviation looking at the color mapping of photos to laser scans (more or less presents depending on methodology) that in certain points can exceed 2 pixels. Therefore, if the matching is solely made on colour information, doesn’t this add errors to matching processses ?

@Götz > Yes. RC can detect each position (single file with let’s say 20 scan pos), and the scan is registered (fixed) and georeferenced. I noticed problems importing huge e57 in RC, but the parts that get imported always worked for me very nicely and smootly. Thus I guess the problem doesn’t come from there ?

For the images, I did not took panoramas. Regular 80+% overlapping with fixed lens. The material is problematic indeed (but as you can guess, quality control over the reconstruction), however each as the maximum number of tie points (wide angle, a lot of features). 

For sure I will try in the coming weeks intensity only, but first I would like to fix and know the reason of fail component matching between coloured scan with internal camera, and fisheye pics (I also have the calibration parameters if needed).

@Support > Any ideas ?

[Anonymous_aa64e24c61]

Hi Florent,

Alright, I just wanted to check because the ordered scan (or rather lack thereof) is often the cause for problems. I agree, this seems to ba ok and not the cause for your problems.

How do you merge your panorama images with the scan if it isn’t the built-in camera?

I would speculate that an error of around 2 pixels is probably too much for good results in RC. You also need to bear in mind, that the laserscan is not absolutely rigid but the surface will be altered as soon as there are enough images providing a different geometry. RC simply transfers the scans into 6 images set up as a cube. That is the reason why it needs reliable RGB information.

Support has not been too active lately so I wouldn’t wait for them too long, but I am always happy to be proven wrong in that respect… 

[Anonymous_154bee8614]

@ Götz > when not using internal camera, I use a device to get the DSLR camera at the same nodal point of the laser scanner (to prevent parallax). However, matching perfectly the image (using SIFT) somehow is made along the horizontal line of the scan point of view and we can therefore notice some deviations sometimes when have a lot of depth in the scene.

[Anonymous_ac400dd84d]

“At the top menu bar, there is a blue menu called “Image” (next to Reconstruction) which only appears when you activate a 2D window”

Strange - this is what I see

 

 

see

[Anonymous_ac400dd84d]

Grrr Edit won’t work. What I see is ‘View’ tab not ‘Image’ - and no button for Tie Points or other ‘what you want to have displayed’. This is the E99 version - perhaps yours is CLI or whatever?

[Anonymous_aa64e24c61]

Huh? Thought I answered…

No, I also have Promo. That’s weird.

Can you post a screenshot with your blue “View” menu active?

Here’s mine: