Hi
We are using RealityCapture for reconstruction, which we then use for Image-Based Rendering. For many scenes this
works well, but in some cases we have a large misalignment between some images and the calibrated cameras.
Ideally we need pixel-accurate alignment.
I am including one example dataset taken with an Iphone 6S. Similar misalignment (worse actually)
occurred with image generated with images from a path-tracer (so a perfect pinhole camera) which we want
to use as ground truth data for testing.
The data is here:
https://filesender.renater.fr/?s=downlo … 68c70af794
(file too large to attach)
Please note that in both cases VSFM+CMPMVS give a perfectly aligned result.
Is there some parameter we need to set during calibration or mesh generation ?
We are happy to wait a bit longer for better camera calibration.
Many thanks in advance.
G. Drettakis
Hi George Drettakis
in what file format (+settings ) are you exporting the data ? and what app used for opening and inspecting exported data ??
Hi
We use default settings and export to bundle.out. All the output is in the zip file.
We use meshlab (which can open a bundle.out file as a “project”) and allows visualization
of the alignment by changing the scroll wheel (it blends between the image and the mesh).
We also use our own IBR software to visualize, and we clearly see the alignment problems
when reprojecting the images onto the geometry. These do not happen with
VSFM+CMPMVS, but the quality of the mesh is of course much better with RealityCapture.
Many thanks in advance for your help.
-g
Hi George Drettakis
Its possible to get the dataset for inspection ? (what export settings have you used for export ? )
if just some images are not bad aligned. Can you try to check to use Maya exported data as we have feedback its working properly…
Hi
The link to the dataset was in the first post, here it is again:
https://filesender.renater.fr/?s=downlo … 68c70af794
The problem is the alignment of the cameras to the mesh. The mesh
itself is fine, no problem there.
This is a bit of a specific request, since we dont use the textured
mesh directly (we reproject the images onto the geometry doing Image-Based Rendering).
Thanks again
Hi George Drettakis
Would be interested why its better to post project the images later and not directly get Tx model ? As it would need for sure some additional cleaning + optimization steps on the reprojected imgs-Tx ??
You send RESULTS, but i need source data (images + RCPROJ data ) to check the alignment and so can testify if export is done the right way or not…
Hi
By the way here are the original images so you can test
https://filesender.renater.fr/?s=downlo … 545fe9684b
Image-based rendering (IBR) is a different way to do high-quality rendering of captured
environments. You can look up our publications if you are
interested:
https://team.inria.fr/graphdeco/publications/
We have several videos of the results with IBR.
Technical/scientific people at your company are probably
aware of these solutions.
Thanks
Hi,
Here is the link to another dataset that it doesn’t align correctly with the mesh. I have included the input images as well.
https://filesender.renater.fr/?s=downlo … 8faa6ca4d7
The parameters that we use are the following:
for the mesh:
ASCII ply with Mov (0,0,0) and Scale (1,1,1)
for the registration:
Bundler v0.3 (negative z) with Mov (0,0,0) and Scale (1,1,1)
fit = Inner region
Resolution = Preserve
Though we have tried using different combinations (inner, outer, in between and preserve and fit) and none of them work.
By setting Undistort images to false, we have noticed that the only difference is that we don’t get new output images, but the bundler file is still using the parameters that are hidden under the “Undistorted images” subcategory.
Thanks.
Hi George Drettakis
Thank you for source images but i need the RCPROJ files too so i can see how it was aligned and etc. + make screenshot of the export parameters used there so its easier to see all the export settings on one image.
RCPROJ_and_folder.png
Thank you for the research stuff, will digg intro that later this week…
Hi,
Okay, I added the rcproj file (it is inside of rc_out folder).
https://filesender.renater.fr/?s=downlo … fdf4cf072b
Thanks again.
Hi George Drettakis
Can you contact me directly on my email ? Have some questions.