Mixed Reality and Spectator View

That’s the main question. Realistically we need something like the Steam VR Unity plugin that outputs multiple layers, so that foreground and background can easily be composited later. A good explanation of the technique is here: