XMP Rotation

Development Asset Creation
,
1

This is a topic widely discussed already, but to no satisfying conclusion.

I am currently working on a script to convert different camera formats, and would like to include reality captures xmp. But I am having some trouble with the way rotations are stored as matrices in xmp. I edited custom xmps with my own rotational and positional values to look at what RC does with them. This is the result:

https://imgur.com/4BFaXmW

As you can see, the rotational values from the xmp matrix do not match the values in RC. It is already documented that RC uses a yxz rotational order here:  https://support.capturingreality.com/hc/en-us/articles/360020679799-How-to-work-with-RealityCapture-registration-export

But the rotational order should not matter anyway as a rotational matrix is of course independent of previous rotational orders. Furthermore, the topic was discussed in the following posts:

https://support.capturingreality.com/hc/en-us/community/posts/360009696080-Different-rotation-of-cameras-in-xmp-and-csv 

https://support.capturingreality.com/hc/en-us/community/posts/360022260791-Wrong-rotation-matrix-from-XMP-export- 

with no result. Can anybody offer some insight which default orientation, coordinate system and rotational conventions the xmp files use?

 

2

Hi BB_99112234

In RealityCapture, Yaw is measured from north. If it is not georeferenced, the Yaw is from the Y axis, which in RealityCapture points up (on the screen).
yaw - an angle of the yaw rotation: the rotation around the yaw (vertical) axis
pitch - an angle of the pitch rotation: the rotation around the pitch (lateral) axis
roll - an angle of the roll rotation: the rotation around the roll (longitudinal) axis

For example:

Yaw = 0, image is oriented to Y (upper side of image is oriented that way)
Pitch = 0, image is looking down
Roll = 0, image is parallel with X axis
So, image is looking down according to RC plane.

If the rotation matrix in XMP looks like this: <xcr:Rotation>-0.600806990019897 0.799386597570746 -0.00346819369376912 -0.210825371930633 -0.162635135620312 -0.963899618846316 -0.771092486861039 -0.578386445454965 0.266243303052733</xcr:Rotation>

then the rotation is stored row wise, i.e.,

R = [-0.600806990019897 0.799386597570746 -0.00346819369376912
     -0.210825371930633 -0.162635135620312 -0.963899618846316
       -0.771092486861039 -0.578386445454965 0.266243303052733 ]

 

3

Thanks for your detailed response! One thing I am struggling with though is the apparent change of matrices from xmp to reality capture. So, when the matrix supplied in xmp looks like this:

<xcr:Rotation> 0 0 1 0 1 0 -1 0 0</xcr:Rotation>, i.e

R =

[0 0 1;

0 1 0;

-1 0 0]

what should correspond to a rotation of 90 degrees around the y axis, the corresponding RC shows:

180 yaw

0 pitch

90 roll

Additionally, the translation does not change, so all the directions show up the same in RC as well in the xmp.

Furthermore, you mentioned yaw meaning around the z-axis, while this post: https://support.capturingreality.com/hc/en-us/articles/360020679799-How-to-work-with-RealityCapture-registration-export  mentions that yaw is a rotation around the y-axis.

What am I missing here? I very much appreciate your help!

4

Hi, I asked our developers about the XMP rotation issue.

For images the yaw is around Z. It is possible, that there is something wrong in that article. But it is true, that the mentioned rotation in the article are true for reconstruction region.

5

Thanks for the quick response. What I still don’t fully understand is the conversion from euler angles (yaw pitch roll in RC) to the rotation matrix in xmp. Could you provide me with the right equations for the conversion?

6

Please note that rotation formats vary. For quaternions, it is not uncommon to denote the real part first. Euler angles can be defined with many different combinations (see definition of Cardan angles). All input is normalized to unit quaternions and may therefore mapped to different ranges. The converter can therefore also be used to normalize a rotation matrix or a quaternion. 

7

I experienced a similar problem (roll-pitch-yaw values not making sense) when attempting to translate the attitude convention I use in my work (direction cosine matrices (DCMs)) to roll-pitch-yaw values to use in RC’s flight log import.  Here’s how I solved this.  

First, note that the forward mapping to produce a DCM from roll-pitch-yaw values is described, for example, here: https://math.stackexchange.com/questions/2796055/3d-coordinate-rotation-using-roll-pitch-yaw.  This post also describes the reverse mapping, that is, extracting roll-pitch-yaw values from a DCM.  

I first ran RC image alignment with only x-y-z flight log information.  I took the posterior roll-pitch-yaw values for a given image and constructed a DCM, call it C_{rc}, using the forward mapping.  I had a prior DCM, C, from my flight log, and assumed the following mapping between them, 

C_{rc} = C_{map} * C.

Solve for the mapping via

C_{map} = C_{rc} * C.’

I’m working with a north-east-down coordinate system, and in my case the mapping was approximately 

C_{map} = [-1, 0, 0 ; 0, 1, 0; 0, 0, -1].

Finally, to set accurate attitude priors in the flight log, I simply premultiply my DCMs by the mapping matrix, 

C_{log} = C_{map} * C,

and decompose C_{log} into roll-pitch-yaw values, which I include directly in the flight log.