We need to warp & blend the output of the final render pass to use with multiple projectors.
If Unreal supports in any way a mode where it renders to a video file without having any windows open, we can leverage that mode to implement warp & blend.
Another way to look at it would be to do the final render to an off-screen render target instead of using the swap chain (back buffer). One way to achieve this would be to have a new Targeted RHI in the Platforms Windows menu for off-screen rendering.
It could be an additional option to an existing RHI that could be activated by a packaging option.
When started with this renderer, the packaged game would run without a visible window (a console window would be ok) but still capture all the inputs from the system. It would render to an off-screen render target.
This renderer would be useful for cloud gaming where a single server can host multiple instances of a game.
Ideally, Unreal Engine would have an NVIDIA Grid SDK NvIFR Target RHI to deploy games on servers.
We would leverage it by doing the render on a server and streaming to the warp & blend system.
NvIFR is a simple SDK for very low latency H264 encode & readback.
If Unreal’s implementation only saves the H264 stream to a raw file that would be fine.
For us, it would be easy to change the file for a network stream.
For other developers that could be a great tool to benchmark and test games at various resolutions and check frame by frame results (there’s a lossless H264 format in Grid SDK).