What are MediaOutputs in nDisplay ?

Development Rendering
, ,
1

I help maintain a plugin where the basic premise is to grab frames from Unreal and display them in our software. However with the release of 5.7 it has stopped working. I’ll go through the flow of what happens and then the title will make sense.

So first the plugin attempts to get the resources via FDisplayClusterViewportProxy::GetResourcesWithRects_RenderThread() which traditionally has no problems.

This calls the impl version which calls FDisplayClusterViewportProxy::GetResourcesWithRects_RenderThread() which is what is failing and resulting in the previous function returning false in our case.

This function has a switch statement depending on the EDisplayClusterViewportResourceType (we’re using InputShaderResource) then calls FDisplayClusterViewportResources::GetRHIResources_RenderThread which brings us to the problem has it tries to iterate over ViewportResources but there is none so it returns false.

So then that brings me to the FDisplayClusterViewport::UpdateFrameContexts() function that appears to be called every frame and it responsible for population ViewportResources. Normally when going into this function this check would pass

if (IsResourceUsed(EDisplayClusterViewportResource::InputShaderResources)) but in the release version of 5.7 it does not.

That is because of this check:

// Resources may be disabled by other rules: media, offscreen nodes, etc.
if (ShouldDisableViewportResources())
{
  return false;
}

And specifically this part inside of it:

// Any other viewports on the headless node that do not use a backbuffer media output.
if (Configuration->IsClusterNodeRenderingOffscreen()
  && !Configuration->GetRenderFrameSettings().CurrentNode.bHasBackbufferMediaOutput)
{
  // Resources should not be allocated for these viewports.
  return true;
}

We are always rendering off screen so the issue in this case is that bHasBackbufferMediaOutput is false. So my main question is what are media outputs and why are they now required for rendering off screen ?

I have also found where this bool is set:

NewRenderFrameSettings.CurrentNode.bHasBackbufferMediaOutput =
        InOutConfiguration.IsMediaAvailable()
        && CfgNodeData->MediaSettings.bEnable
        && CfgNodeData->MediaSettings.IsMediaOutputAssigned();

IsMediaOutputAssigned() returns false because there are no valid MediaOutputs in the MediaSettings. Not sure what these media outputs are or how to set them up so any information would be appreciated.

Here is the commit where ShouldDisableViewportResources() was introduced: https://github.com/EpicGames/UnrealEngine/commit/fd10a6f7ccdee898e53fc73713e9c263193360fe

[Attachment Removed]

2

Hi Josh, it could be a regression. I will be back soon with more details on this one

[Attachment Removed]

3

Josh, this is indeed a regression on our end. We hope to roll out the fix in 5.7.1, and I will follow up with a potential fix as soon as possible for you.

[Attachment Removed]

4

No ETA yet, however, as a workaround, could you please add dummy media output to node on your end? That should pass the checks and unblock the rendering.

[Attachment Removed]

5

Media Outputs are the canonical way of exporting nDisplay renders. Can be use to export icvfx cameras, viewports, and backbuffer textures mostliy over ST-2110 and SharedMemoryMedia, sometimes to render targets or other types of media outputs.

[Attachment Removed]

6

Josh,

Would that workaround with adding dummy Media Output work on your end for 5.7?

Thanks!

vitalii

[Attachment Removed]

7

Great thank you

[Attachment Removed]

8

Any idea when 5.7.1 might be released ?

[Attachment Removed]

9

Ya that works thanks. What are MediaOutputs though ? Were they not intended to be required ?

[Attachment Removed]

10

Yep that works

[Attachment Removed]