CGI Renderings > XMP Creation > RealityCapture > Postshot [Gaussian Splat Model Workflow]

indexrefraction1 · May 13, 2025, 2:35pm

Hi, I’m trying to create a Gaussian Splat model from CGI rendered images.

In my test I render about 200 images at various positions and viewing directions.
I also generate an XMP for each image containing the focal length, the position and rotation matrix.

Now I run RealityCapture by CLI command to align the image.
It is basically the workflow demonstrated here, just not using blender.

This generates a CSV with the camera positions and a PLY point cloud model.

Problem is that the CSV only contains 12 cameras instead of the 200 it was supplied with.

Dragging the folder into Postshot i can’t see the exact cameras as shown in that video,
but only a few and the training output isn’t usable at all.

Does anybody maybe have an idea why RealityCapture doesn’t take all given cameras into account?

OndrejTrhan · May 13, 2025, 3:08pm

Hi @indexrefraction1
There is no big visibility of RealityCapture in your tutorial.
Is it possible that you’ve got more components after the alignment or that you have had selected some images before exporting the CSV?
What were you alignment settings and then export settings?

indexrefraction1 · May 13, 2025, 8:12pm

hm its hard to even find the right questions about this …

the video shows a workflow starting with blender and i try to recreate this using cinema4d:

render images
create XMPs for all frames/cameras (from within blender or now cinema4d)
run RealityCapture using a batch script (attached below)
to align the images and store a camera csv and sparse model ply
run Postshot on the folder to train a gaussian splat model.

i suspect that RealityCapture does either not use the XMPs at all
or that there is some other reason that not all cameras are used.

is there a way to force RealityCapture to use/keep all provided images/XMPs ?
or is RealityCapture always making a selection?
are there reasons XMPs could be ignored?

@OndrejTrhan:
RealityCapture is just run by the script below, I’m not doing anything from within the app. When i reopen the project file i see that some components were created.

The model is a train interior with a lot of resembling images. so using all images with their XMP information is vital.

for testing i use low res (2400x1600px) renderings that are a bit noisy
the cameras are evenly spread around the wagon, similar to the video.

@echo off

:: Change directory to given images folder (1st script argument)
pushd "%1"

set "RealityCaptureExe=C:\Program Files\Capturing Reality\RealityCapture\RealityCapture.exe"

set "Images=%CD%"
set "ModelName=SplatModel"

set "ExportRegParams=%CD%\ExportRegParams.xml"
> "%ExportRegParams%" (
    echo ^<Configuration id="{2D5793BC-A65D-4318-A1B9-A05044608385}"^>
    echo   ^<entry key="calexTrans" value="1"/^>
    echo   ^<entry key="calexHasDisabled" value="0x0"/^>
    echo   ^<entry key="MvsExportScaleZ" value="1.0"/^>
    echo   ^<entry key="MvsExportIsGeoreferenced" value="0x1"/^>
    echo   ^<entry key="MvsExportIsModelCoordinates" value="0"/^>
    echo   ^<entry key="MvsExportScaleY" value="1.0"/^>
    echo   ^<entry key="MvsExportScaleX" value="1.0"/^>
    echo   ^<entry key="MvsExportRotationY" value="0.0"/^>
    echo   ^<entry key="MvsExportcoordinatesystemtype" value="0"/^>
    echo   ^<entry key="MvsExportNormalFlipZ" value="false"/^>
    echo   ^<entry key="MvsExportRotationX" value="0.0"/^>
    echo   ^<entry key="hasCalexFilePath" value="1"/^>
    echo   ^<entry key="MvsExportNormalFlipY" value="false"/^>
    echo   ^<entry key="MvsExportNormalSpace" value="Mikktspace"/^>
    echo   ^<entry key="calexHasUndistort" value="-1"/^>
    echo   ^<entry key="MvsExportNormalFlipX" value="false"/^>
    echo   ^<entry key="MvsExportRotationZ" value="0.0"/^>
    echo   ^<entry key="calexFileFormat" value="Internal/External camera parameters"/^>
    echo   ^<entry key="MvsExportMoveZ" value="0.0"/^>
    echo   ^<entry key="calexFileFormatId" value="{0CA18733-1EBC-4254-9974-17197EB409BD}"/^>
    echo   ^<entry key="hasCalexFileName" value="1"/^>
    echo   ^<entry key="calexHasImageExport" value="-1"/^>
    echo   ^<entry key="MvsExportMoveX" value="0.0"/^>
    echo   ^<entry key="MvsExportNormalRange" value="ZeroToOne"/^>
    echo   ^<entry key="MvsExportMoveY" value="0.0"/^>
    echo ^</Configuration^>
)

set "ExportSparsePointCloudParams=%CD%\ExportSparsePointCloudParams.xml"
> "%ExportSparsePointCloudParams%" (
    echo ^<Configuration id="{2D5793BC-A65D-4318-A1B9-A05044608385}"^>
    echo   ^<entry key="calexTrans" value="1"/^>
    echo   ^<entry key="bAscii" value="false"/^>
    echo   ^<entry key="calexHasDisabled" value="0x0"/^>
    echo   ^<entry key="bVertexColor" value="true"/^>
    echo   ^<entry key="MvsExportScaleZ" value="1.0"/^>
    echo   ^<entry key="MvsExportIsGeoreferenced" value="0x1"/^>
    echo   ^<entry key="MvsExportIsModelCoordinates" value="0"/^>
    echo   ^<entry key="MvsExportScaleY" value="1.0"/^>
    echo   ^<entry key="MvsExportScaleX" value="1.0"/^>
    echo   ^<entry key="MvsExportRotationY" value="0.0"/^>
    echo   ^<entry key="MvsExportcoordinatesystemtype" value="0"/^>
    echo   ^<entry key="MvsExportNormalFlipZ" value="false"/^>
    echo   ^<entry key="MvsExportRotationX" value="0.0"/^>
    echo   ^<entry key="hasCalexFilePath" value="1"/^>
    echo   ^<entry key="MvsExportNormalFlipY" value="false"/^>
    echo   ^<entry key="MvsExportNormalSpace" value="Mikktspace"/^>
    echo   ^<entry key="calexHasUndistort" value="-1"/^>
    echo   ^<entry key="MvsExportNormalFlipX" value="false"/^>
    echo   ^<entry key="MvsExportRotationZ" value="0.0"/^>
    echo   ^<entry key="calexFileFormat" value="Sparse point cloud as Polygon File Format (*.ply)"/^>
    echo   ^<entry key="MvsExportMoveZ" value="0.0"/^>
    echo   ^<entry key="calexFileFormatId" value="{B63136B7-2E64-4D08-B5B1-A945F1AED679}"/^>
    echo   ^<entry key="hasCalexFileName" value="1"/^>
    echo   ^<entry key="calexHasImageExport" value="-1"/^>
    echo   ^<entry key="MvsExportMoveX" value="0.0"/^>
    echo   ^<entry key="MvsExportNormalRange" value="ZeroToOne"/^>
    echo   ^<entry key="MvsExportMoveY" value="0.0"/^>
    echo ^</Configuration^>
)

set "ExportReg=%CD%\%ModelName%_cameras.csv"
set "SparsePointCloud=%CD%\%ModelName%_sparse.ply"
set "Project=%CD%\%ModelName%.rcproj"

if exist "%ExportReg%" del "%ExportReg%"
if exist "%SparsePointCloud%" del "%SparsePointCloud%"

"%RealityCaptureExe%" -addFolder "%Images%" ^
-set "sfmImagesOverlap=Low" ^
-set "sfmCameraPriorWeight=1000.0" ^
-set "sfmCameraPriorWeightOrientation=1000.0" ^
-set "sfmCameraPriorAccuracyX=0.0" ^
-set "sfmCameraPriorAccuracyY=0.0" ^
-set "sfmCameraPriorAccuracyZ=0.0" ^
-align ^
-setReconstructionRegionAuto ^
-save "%Project%" ^
-exportRegistration "%ExportReg%" "%ExportRegParams%" ^
-exportSparsePointCloud "%SparsePointCloud%" "%ExportSparsePointCloudParams%" ^
-quit

OndrejTrhan · May 14, 2025, 6:56am

Hi,
are you sure that the XMPs are created properly? Also, in your case, are the saved as draft, exact or locked? When you import the image folder with XMP to RealityCapture, what do you see as a prior pose?
You also wrote, that you see some components in RealityCapture. It looks to me, that only the first component with 12 images was exported.

indexrefraction1 · May 14, 2025, 4:25pm

thanks a lot for your support !

attached is is a sample XMP
the only thing set is the FocalLength35mm, Rotation, Position and DistortionCoeficients,
the rest is static and the same for all images/cameras
begin=“” contains the bytes 0xfeff, where the id=“…” is coming from is unknown,
i just took the same code for this as in the video.

even if the coordinate system would be different between the apps,
the cams/model should generate (with a wrong orientation maybe),
so i guess that is not the problem.

attached a screenshot after reopening the project saved by the script.
there are multiple components but a lot of cams have been ditched.

could it be a problem of the cameras?
i’m using about 15-20 icosaeders (diameter ~60cm)
with 12 cams each facing inwards.

<?xpacket begin="" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description
   xcr:Version="3"
   xcr:PosePrior="initial"
   xcr:Coordinates="absolute"
   xcr:DistortionModel="brown3"
   xcr:FocalLength35mm="28.000000000000000"
   xcr:Skew="0"
   xcr:AspectRatio="1"
   xcr:PrincipalPointU="0"
   xcr:PrincipalPointV="0"
   xcr:CalibrationPrior="initial"
   xcr:CalibrationGroup="-1"
   xcr:DistortionGroup="-1"
   xcr:InTexturing="0"
   xcr:InMeshing="1"
   xmlns:xcr="http://www.capturingreality.com/ns/xcr/1.1#">
   <xcr:Rotation>1.000000000000000 0.000000000000000 0.000000000000000 0.000000000000000 0.525731112119135 0.850650808352039 -0.000000000000000 0.850650808352039 -0.525731112119135</xcr:Rotation>
   <xcr:Position>-0.000000000000000 -15.166976882251152 -2.639272571380356</xcr:Position>
   <xcr:DistortionCoeficients>0.0 0.0 0.0 0.0 0.0 0.0</xcr:DistortionCoeficients>
  </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>

OndrejTrhan · May 15, 2025, 6:56am

I’ve checked your XMP on my image and it worked.
What I noticed, you are using PosePrior set as “initial”. What you can try is set it to “locked”.
Also, the problem with the cameras could be in a wrong overlap.
Are you able to align those images without the XMP usage?
And as your images are georeferenced in local system, you can use the setting Merge georeferenced components. You can find it in the Advanced Alignment settings.

indexrefraction1 · May 15, 2025, 7:52am

thanks otrhan

Are you able to align those images without the XMP usage?

no, but i expected that, because in train interiors a lot of images are resembling,
with that cgi precision they can be even identical for some compartments.

What I noticed, you are using PosePrior set as “initial”. What you can try is set it to “locked”.

ok wow, i used xcr:PosePrior=“locked” and cr:CalibrationPrior=“locked” as well,
and now all the cameras are there! thanks a lot for this!

And as your images are georeferenced in local system, you can use the setting Merge georeferenced components. You can find it in the Advanced Alignment settings.

is it possible to set that by script as well?
but maybe its not needed anymore now, anyway

i’ll need to check now… the positions seem ok now,
but the orientation of the cameras is off
so there is some coordinate system translate issue as well.

is there any place the RealityCapture coordinate system is documented ?
Is that … Right Hand … Z-up, +Y forward ?
and the units are meters in RC, right?

OndrejTrhan · May 15, 2025, 8:29am

Yes, it is possible to set using the -set command.

There are some places, where you can find some information:

And the basic units are in meters.

indexrefraction1 · May 15, 2025, 4:59pm

ah well, i wasted a full day on this… .-)
seems like a simple Y / Z axis swap, but the cam rotations are weirdly off …
needs a fresh head .-)

thanks a lot for your help till now, otrhan !
i’d never have figured that out …

indexrefraction1 · May 22, 2025, 8:19am

@OndrejTrhan

thanks for helping me with this.
after some days i finally found the correct translation from c4d to reality capture.

but if i may ask a follow up question :
i’m now able to do the same as shown in the video above
but my splat model quality is quite bad.

i m testing on a train car interior. i’m using a dodecaeder globe with 12 sides and one camera pointing in the (inwards) face normal. the diameter is about 40-50cm and i put one of these every 90cm along the aisle.
i rendered with 2.4x1.6k in low quality which is a bit noisy.

without experience its is hard to judge what is the main reason for the bad result i get in postshot.
but already in reality capture i notice that the generated ply looks very randomly, i’m getting a blob of points and nothing showing the train wagons interior geometry.

so… is there a way to increase the sparse point clouds quality by settings?
or is this just an issue of the input quality (not enough cameras, render quality)

best, index

OndrejTrhan · May 23, 2025, 7:29am

Hi, unfortunately there is no such settings.
But as you are also getting a bad model, there could be some issues with used images. The resolution is really small for such work.

indexrefraction1 · May 23, 2025, 6:17pm

below is what i get in RealityCapture and Postshot
its not just bad, the train wagon is not even recognizable in the point cloud

atm, i wonder if something else could be wrong
the focal length should be correct, though

Postshot limits the resolution to 1600px by default,
but RealityCapture seems to prefer larger images (?)
Could it be that it is mainly the noise?

I probably have to test in a small area with

better render quality
higher resolution
more than 12 cameras per location
more locations in Y direction
additional locations Z direction

sadly with the needed render times
highres and quality and more cameras is going to be an issue

indexrefraction1 · May 23, 2025, 6:18pm

OndrejTrhan · May 26, 2025, 6:54am

Sometimes the “forced” alignment is bad for RealityCapture.
Have you tried to import the generated images without XMPs? Are you able to align those images?

indexrefraction1 · June 8, 2025, 8:57am

hi, in the mean time i made some tests with

better render quality (less noise)
higher resolution
more cameras
not using the XMPs at all

but none made it really better…
so i’m starting to think that maybe the focal_len_35mm value might be incorrect

below is the code i’m using to calculate it
it is set in the XMP as xcr:FocalLength35mm=“36.0”

does that make sense or is it maybe focal_len_35mm = 1.0 in this case ?

there is some info here about this

def CameraParameters(cam, rd=None):
    xres_px = rd[c4d.RDATA_XRES_VIRTUAL] # 3000
    yres_px = rd[c4d.RDATA_YRES_VIRTUAL] # 2000
    aspect_ratio = rd[c4d.RDATA_FILMASPECT] # 1.5

    focal_len_mm = cam[c4d.CAMERA_FOCUS] # 36.0
    sensor_w_mm = cam[c4d.CAMERAOBJECT_APERTURE] # 36.0
    sensor_h_mm = sensor_w_mm / aspect_ratio # 24.0

    shifting_x = cam[c4d.CAMERAOBJECT_FILM_OFFSET_X] # 0.0
    shifting_y = cam[c4d.CAMERAOBJECT_FILM_OFFSET_Y] # 0.0

    focal_len_px = (focal_len_mm / sensor_w_mm) * xres_px 

    return (focal_len_px, shifting_x, shifting_y,
            focal_len_mm, sensor_w_mm, sensor_h_mm)

(focal_len_px, shifting_x, shifting_y,
    focal_len_mm, sensor_w_mm, sensor_h_mm) = CameraParameters(cam)

# 3000.0, 0.0, 0.0, 36.0, 36.0, 24.0

crop_factor = 36.0 / sensor_w_mm 
focal_len_35mm = focal_len_mm * crop_factor

# crop_factor = 1.0
# focal_len_35mm = 36.0

OndrejTrhan · June 9, 2025, 6:53am

Hi,
I suppose it is 36 mm in your case.
Are you able to align the images without XMP usage? If not, there could be the problem of camera positions and the overlap. Using the dodecaeder globe positions maybe isn’t ideal for photogrammetry of the interior.

indexrefraction1 · June 10, 2025, 9:25pm

Hi !

After researching deeper I think the FocalLength35mm is calculated correctly.

Without XMP i don’t get a better result.

I changed to Icosaeder with 20 cameras already.
Also I use a focal length of 24mm to get more overlapping.

Still.. with 20 cameras pointing inwards on each side of an icosaeder (radius 30cm) you get quite a lot of cameras covering each point. It surprises me that the quality is that low.

Next thing to try is maybe a really high resolution to see if that is the issue.

In the video the cameras are pointing inwards and outwards, so cameras are doubled. i didn’t try that yet, because my Icosaeder size is limited to the train aisle width and pointing outwards would be extremely close to the seats for some images.

maybe the relatively small size of the icosaeder (radius 30cm) is also an issue.
in the video the camera sphere is much larger.

OndrejTrhan · June 11, 2025, 6:52am

But is it possible to align the images properly without XMPs?

indexrefraction1 · June 12, 2025, 11:48am

Without XMP RC doesn’t seem to properly find the camera positions

this all odd…
i tried highres renderings ~8k for one icosaeder
i tried doubling the cameras to point in + outwards on each surface
i tried prior=initial

nothing works

… is it an option that somebody at Epic/RC takes a look at this?
(using my images + XMPs + script)

OndrejTrhan · June 12, 2025, 12:08pm

As the images are not aligned without XMPs then there is no enough overlap between the images.
How many features are found over the images during the alignment process?
For the interiors it is better to capture the images as:

If you wish you can share the data with us. I will send you the invitation for the data upload.