Reconstruction Region Set but not applied

Dani_rm · November 15, 2024, 11:20am

Hello,
we are using the -setReconstructionRegion command to set the bounding box.
Unfortunately for some cases the processed model results in a model that is too big and has blurry outlines.
When I checked the .rcbox file it has the correct width/height etc. Also sometimes if we retrigger the exact same pipeline the reconstruction region was set correctly and has the correct size.
As you can see in the images the models are blurry on the outside and the surface is at least twice or three times the size we specified in the rcbox file.

Do you know what would cause such a situation, or what we could do to prevent it?

Thanks in Advance!

OndrejTrhan · November 15, 2024, 11:53am

Hi Daniele
what is the script you are using? This looks like region is not loaded at all.

Dani_rm · November 22, 2024, 4:45pm

Thanks for your response. We run a custom custom script that provides a .rcbox file to the -setReconstructionRegion command.
But the same command with the same file sometimes doesn’t actually set the reconstruction region. When I checked the Reality Capture UI while the automated script was running, I noticed that the reconstruction region was never set. Even if we saw that the -setReconstructionRegion ran and no error was thrown.

Is it possible to check whether the reconstruction region was successfully set via a cli command?

OndrejTrhan · November 25, 2024, 8:09am

Hi Daniele,
how was the reconstruction region created?
Is it possible to import that region manually?

It is not possible to check that using the CLI command.

MatejAirteam · January 14, 2025, 3:56pm

Hi Ondrej, I work at the same company that Daniele worked at, and would like to continue since we’re still facing the issue.

Our flow when applying the rcbox file is the following:

Align the images, generate and export the reconstruction region
Get the elevation and use GPS coordinates to create a new rcbox file
Use the newly created rcbox to set the region

Here is the content of the file:

'<ReconstructionRegion globalCoordinateSystem="+proj=longlat +datum=WGS84 +no_defs" '
        'globalCoordinateSystemName="epsg:4326 - GPS (WGS 84)"\n'
        '   isGeoreferenced="1" isLatLon="1" '
        'widthHeightDepth="48.0641059874739 53.278949737143 10.1386680599677">\n'
        '  <globalCoordinateSystemWkt>GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",'
        '6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]'
        "</globalCoordinateSystemWkt>\n"
        f"  <yawPitchRoll>0.0 0.0 -{yaw}</yawPitchRoll>\n"
        f"  <widthHeightDepth>{depth_west} {width_north} {height}</widthHeightDepth>\n"
        '  <Header magic="5395016" version="2"/>\n'
        "  <CentreEuclid>\n"
        f"    <centre>{long} {lat} {z}</centre>\n"
        "  </CentreEuclid>\n"
        '  <Residual R="1 0 0 0 1 0 0 0 1" t="0 0 0" s="1" '
        'ownerId="{8D3C7DA5-CD49-417A-8EC0-34C096F0B330}"/>\n'
        "</ReconstructionRegion>\n"

As Daniele said, the weird thing is that if we rerun the same project for a 2nd or 3rd time, the BB will be there and it’ll work. This happens for a wide variety of projects and I couldn’t find any correlation between them. We also can’t do this manually.

I do have additional questions:

Is it possible for the bounding box to fail when it’s being applied but for no error to be raised by the command?

OndrejTrhan · January 15, 2025, 9:59am

Hi Matej,
this is quite unusual workflow.
Are you using that region in the same project? Do you have set the project and output coordinate system properly? How are you getting the elevation? Are you changing that value in the rcbox file?

Also, I exported the rcbox file and checked it and it is different to yours. Are you adding some lines into that file? As you have widthHeightDepth line twice…
Also, for the centre, are you using the degree coordinates or radians?

As you are running it more times, is that a fresh start of the project?

Regarding the manual testing, I was curious, if it is possible to import the rcbox into your project using the Import Reconstruction Region tool.

I suppose there is no error for the importing of the region in the system.

MatejAirteam · January 16, 2025, 10:14am

Hi Ondrej,

It seems I gave you some wrong information at the start. I’m not familiar much with RC and was just starting to debug this so sorry for that.
It seems this bug is always constant so it won’t fix by itself after the 1st or 2nd try. I’m guessing that the problem here is with the RC box file.

Regarding the elevation, we parse the rcbox file and read the last value of the CentreEuclid tag. We the increase it by 10 (constant) and set it in the new rcbox file along with the other values such as yaw, width, height, depth, long and lat.
Would RC throw an error if the rcbox file is not formatted correctly or has some incorrect values?

OndrejTrhan · January 16, 2025, 11:25am

Hi Matej,
can you share the original and modified rcbox file? Also some images from that area?

Yes, if the values are incorrect, it will throw an error.

MatejAirteam · January 16, 2025, 4:28pm

Here’s an example of the rcbox file

<ReconstructionRegion globalCoordinateSystem="+proj=longlat +datum=WGS84 +no_defs" globalCoordinateSystemName="epsg:4326 - GPS (WGS 84)"
   isGeoreferenced="1" isLatLon="1">
  <globalCoordinateSystemWkt>GEOGCS["GCS_WGS_1984",DATUM["D_WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]]</globalCoordinateSystemWkt>
  <yawPitchRoll>0.0 0.0 -57.994596645808386</yawPitchRoll>
  <widthHeightDepth>23.19294638093561 31.892667421139777 60</widthHeightDepth>
  <Header magic="5395016" version="2"/>
  <CentreEuclid>
    <centre>0.2872301359801154 0.8349964618725966 123.650970031507</centre>
  </CentreEuclid>
  <Residual R="1 0 0 0 1 0 0 0 1" t="0 0 0" s="1" ownerId="{8D3C7DA5-CD49-417A-8EC0-34C096F0B330}"/>
</ReconstructionRegion>

I tried importing it manually by going to Mesh Model → Reconstruction Region in the Import section and then selecting this rcbox file but nothing happens.

Here’s an image:

OndrejTrhan · January 17, 2025, 8:08am

Hi Matej, thank you for your file.
I tested it and it looks I am able to import your region (I used original one and also changed the values for my project).
Can you check your Project and Output coordinate system settings?
Also, are the images from the attached picture georeferenced (because I don’t see any orange lines for the images)?
Can you try the application reset (Reset RealityCapture) and then try the import again?

MatejAirteam · January 17, 2025, 11:40am

The coordinate system on the Project and the Output is epsg:4326 - GPS (WGS 84).
The component is not Geo-references but I am not sure how or whether there exists a setting for each image.

I’m not sure if resetting RC will do anything since other projects have been processed with a BB.

OndrejTrhan · January 17, 2025, 12:03pm

OK, the usage of different coordinate systems could be a problem there.
Can you show me also the 1Ds view with one selected input image in the component?
As the reconstruction region is georeferenced, also the component must be georeferenced. As it is not for you, it is placed in different position, which is far away from the component and therefore it is not imported.

I tested this on my dataset, turned off the camera priors and nothing happened. To use the reconstruction region I had to turn on the priors and create georeferenced component.

The setting for separated images is in 1Ds view after image selection:

For all images it is possible to disable the Camera priors settings under the Alignment settings:

The reset can help turn the app back to its original state when some wrong settings were used, but it is hard to find them.

MatejAirteam · January 17, 2025, 4:33pm

It seems the no Georeference components was my mistake since my local testing env wasn’t the same as production. I’ve changed it and it now applies the BB locally which is even stranger since it’s still not working in production for some buildings.
I’m not quite sure how to continue, do you have an idea of what might go wrong in those cases?

Can you let me know why are the coordinate systems wrong since they both match and are the same as the one in the rcbox file?

OndrejTrhan · January 20, 2025, 7:55am

Hi Matej,
as the region is now working for you I suppose it is better, as it wasn’t working at all.
I suppose that in the production project will be some similar error. Is it possible to share the failing project with its reconstruction region with us?

RealityCapture uses different coordinate systems. Project shows the coordinate’s values in chosen system, but if the Output system is set to different one, you will see those values as exports.
For the reconstruction region, as it was created from georeferenced data, it says so in the file. But it can be used also for local system, which uses georeferenced data and the coordinates are different to WGS. In your case, you used the georeferenced region for not georeferenced component. The component in this case is close to system’s origin and the values for region are too big to be imported (or something like that).

MatejAirteam · January 20, 2025, 10:43am

You’re correct as I’m using the local coordinate system of the model to set the BB, but It’s still not clear to me what changes need to be made to the rcbox file in order to let RC know this?

OndrejTrhan · January 20, 2025, 11:02am

As you’re using local system, are you sure that all projects are in the same local system?
In that case also the reconstruction region should be exported in the same local system. But (according to your shared data) it was created for georeferenced component, so also the component where you want to use that region should be georeferenced.

RCBOX file for not georeferenced component looks like this:

<ReconstructionRegion globalCoordinateSystem="NONE" globalCoordinateSystemWkt="NONE" globalCoordinateSystemName="NONE"
   isGeoreferenced="0" isLatLon="0" yawPitchRoll="0 -0 -0">
  <widthHeightDepth>72.5264205932617 50.7648887634277 44.0060348510742</widthHeightDepth>
  <Header magic="5395016" version="2"/>
  <CentreEuclid>
    <centre>2.53373622894287 -11.9795780181885 21.0632400512695</centre>
  </CentreEuclid>
  <Residual R="1 0 0 0 1 0 0 0 1" t="0 0 0" s="1" ownerId="{776E95B5-C5C7-49E3-94C9-494C26160452}"/>
</ReconstructionRegion>

For local coordinates, are you keeping the image’s geoinfo or are you turning it off? And for local system, do you have some GCPs to get the component to the wanted position? As without knowing this, the component could be on slightly different positions for each alignment, as there is no information about where to place the component without those data.

MatejAirteam · January 20, 2025, 12:07pm

We’re applying the BB based on some GPS coordinates we have. The process is that we’re converting these GPS coords to find the smallest possible rectangular containing them in order to then set this as a BB. The resulting values are put in the rcbox file and this is always relative to the local coordinate system. The component though is georeferenced, which I guess is because of the true value for isGeoreferenced in the rcbox file.

But the problem for me now, why does it not work on the 1st try but works on 2nd or 3rd one? For some other projects, this works on the 1st try so I’m guessing RC is converting the values correctly. Can there be an error in the conversion if it happens to soon after alignment or? Also, the 2nd and 3rd try are started from scratch, just FYI.

Any pros/cons that you could think of on using the WGS versus the local system?
I think at the moment we’d want to keep the images geoinfo and we also don’t have any GCPs. I’m guessing we would have to apply these if we want to use the local system in order to get the correct BB? Also, do we need these if we’re using WGS instead?

Also, thank you for the help so far, RC is a very new thing to me.

OndrejTrhan · January 20, 2025, 12:59pm

How are you getting those coordinates? Are you just rewriting the RCBOX file or are you adding some lines there?
You can check if the component is georeferenced in the application in 1Ds view next to the component name. If it is you’ll see the georeferencing icon there (a pin). isGeoreferenced value is valid only for the rcbox file (how it was created).

What is the difference between 1st and 2nd try? As you are writing it is started from scratch, there has to be something different in the way of the process… The best option there will be to share your project with us (the failing one) also with the original and modified rcbox. Could it be possible?
If everything is set up properly, then the conversion should be applied.

I think the local system should be more readable (as it in metric system, mostly). But in general, it doesn’t matter what kind of system are you using.
I think it is not necessary, but using GCPs you can get more precise models (as geoinfo from drones is not so precise).

MatejAirteam · January 20, 2025, 1:25pm

Regarding the project sharing, let me check with my team on whether this is possible and I’ll let you know. While we’re on that topic, is it possible to set up a call whenever you’re available?

Regarding the GPS coordinates, we get them from Google maps where the end-user marks the building so that we can only process it and not the surroundings. We then use this data to calculate the bounding box (as a rectangle) and set the coordinates in a new rcbox file which we use to apply the BB. Regarding the 1st and 2nd try, strangely there is no difference and is processed with the same exact data.

OndrejTrhan · January 20, 2025, 1:49pm

I am sorry, but we don’t do calls. Only written help is available there.

As you are getting the coordinates from Google maps, it is possible that the data won’t be so precise. Also, there could be a difference between the model height computed from drone’s data and the height obtained from the maps… I suppose it is not an ideal workflow, as there is a place for quite a lot of mistakes.