RealityCapture XMP Camera Math
Background
There are different camera models in RealityCapture, but you can specify, which you want to use after all cameras are registered (aligned). This should be called as a final mod…
https://dev.epicgames.com/community/learning/knowledge-base/vzwB/capturing-reality-realitycapture-xmp-camera-math
In the Pixel Coordinates section, the method you mentioned:
scale = max( image width, image height )
px = scale*m[0] + image width / 2
py = scale*m[1] + image height / 2
seems like it should instead be:
scale = min( image width, image height )
px = scale*m[0] + image width / 2
py = scale*m[1] + image height / 2
Using minimization worked really well in my case.