Is there a way to get LDR Scene Color and Alpha in a Single SceneCapture pass?

Development Rendering
, ,
1

Is there a way to get an LDR Scene Capture with Alpha, without paying the costs for two separate Scene Capture passes?

In our game, we use SceneCaptureComponents to capture dynamic in-game objects for use in the UI. To get an image that looks appropriate, we currently must do two SceneCaptures, one for the LDR color, and a second for the Alpha. This has both a significant CPU cost, as well as has memory costs for data we do not use (the color results of the Alpha capture)

We do this because to get a color result that looks correct, we must use one Component with Capture Source set to `Final Color (LDR) in RGB`. And to get an alpha that looks correct with things like VFX, we must capture again with `Scene Color (HDR) in RGB, Inv Opacity in A`. We could not find a built in Capture Source that provided both the LDR color and the Opacity.

We were hopeful that there would be a way to do this capture in a single pass to improve both CPU and Memory usage.

We were hopeful that some option such as these might exist:

  • A) A way, possibly through engine modification, that would allow a CaptureSource like `Final Color (LDR), Inv Opacity in A`
  • B) A way to reliably apply the tonemapping operations to take an HDR image down to LDR, so that we can use `Scene Color (HDR) in RGB, Inv Opacity in A` and apply that operation.
  • C) Alternate optimizations that might meet those goals, such as a way to optimize the Show Only list and not pay the memory for the unused color channels of the Alpha capture
    • In our earlier testing, it seemed that the ShowOnly list was applied as a filter on the whole scene, and Scene Captures still had significant CPU costs even when only one item was in the ShowOnly list if the main World was complex enough.
  • D) A canonical way to show a model in the UI without needing to use a SceneCapture
  • E) Some other recommendation?

We are highly motivated to take effort on our side to improve the performance issues we are seeing, but we are not sure what direction is most advisable. Is there a good way to for us to eliminate the extra overhead we are seeing?

2

Steps to Reproduce

  1. Open the attached project
  2. Open DemoMap
  3. Inspect BP_TwoPassCapture and note that `CaptureScene` must be called twice
  4. Hit Play In Editor
  5. Note that a sphere shows in the right side of the UI that looks approximately like the sphere scene in world

Ideally the final image in Step 5 would be possible without needing the two Captures in Step 3

3

Adding reply with demo project attached, for visibility for other UDN members browsing this question.

4

Greetings,

Alpha can be enabled for the LDR final output texture using “Alpha Output” in Project Settings, under Engine -> Rendering -> Default Settings. The project setting controls the default value of r.PostProcessing.PropagateAlpha CVar, which can also be used to toggle alpha without needing to restart the engine. A slight downside of this setting is that it is global -- it can’t be enabled for individual views or scene captures. There is a performance cost of the setting, especially for TSR. You can disable alpha for TSR by setting the CVar r.TSR.AlphaChannel to 0, assuming your Scene Capture doesn’t use TSR (it wouldn’t if you are going for performance).

An option similar to “C” (faster rendering of a standalone alpha channel) is possible in UE 5.6, using a Custom Render Pass. Custom Render Passes only render the base pass, skipping shadows, lighting, and post processing, and are 10 times faster than a full Scene Capture on the CPU. UE 5.6 added support for translucency to Custom Render Passes. You can experiment with a Custom Render Pass by choosing Scene Color with Inv opacity, and setting “Render in Main Renderer” (you should see a visual change as lighting gets disabled).

If you want to get even better performance, you could use such a Custom Render Pass to achieve option “D”, by authoring a single pass material for your UI objects that includes forward lighting and tone mapping built into it. Unfortunately, the engine doesn’t currently provide an out-of-the-box solution for this, so you would need to implement that all yourself. It would be possible to add self-shadows to a UI object using an additional depth Custom Render Pass and the experimental shadow PCF material functions provided in the engine here: Content\Functions\Engine_MaterialFunctions03\Shadows\ShadowPCF8x8.uasset. There’s not an official example in the engine, but here’s a screenshot of a portion of a material graph that uses it:

[Image Removed]The Texture Object is a depth Scene Capture (with Render in Main Renderer set, so it runs as a Custom Render Pass), and it uses a Material Parameter Collection that looks like this to make the transform of the Scene Capture available in the material graph:

[Image Removed]And you’d set the following Collection Transform properties in the shadow map Scene Capture, to tell it to write to the Material Parameter Collection:

[Image Removed]Those screenshots should show enough information to make it work. It’s experimental at this stage due to the steps required -- in the future it should require fewer steps. Material Functions don’t support Material Parameter Collection parameter references as an input type, so the inputs needed by PCF have to be computed by the user outside the Material Function, which is where the complexity comes from. And you’d need to solve lighting and tone-mapping before worrying about shadows. Anyway, apologies for the tangent if you don’t end up going that route, but I wanted to provide all the options!

Cheers!

--Jason Hoerner

5

Thanks for such a detailed answer Jason!

It looks like we should be able to get everything working with this information. On first tests it looks like r.PostProcessing.PropagateAlpha will allow us to bring things down to a single pass.

We are not yet on Unreal 5.6, but once we are I will try the additional suggestions, they sound great, thank you!

Cheers!

-Rick