Mixed Reality and Spectator View

We record the headset and controller transforms as well as the input buttons every frame. Then save that out to a file. We can then replay the interaction after the fact.

We have used it to generate 360 stereo video, but it would work fine for being able to see the playback overlaid on video if you had a camera setup to do it.