Issues of scene capturing using Vulkan

[Okaminu (Okami)]

Hi Epic,

So I’ve been trying the scene capturing using USceneCaptureComponent2D and UTextureRenderTarget2D with Vulkan both on Windows & Linux in UE 4.26. There are a couple of issues as follows:

(1) The captured image looks much darker than running with In-Editor/Standalone. I followed the settings in here:
https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Plugins/Experimental/PanoramicCapture/Source/PanoramicCapture/Private/SceneCapturer.cpp#L89
Specifically, I’m setting CaptureSource as SCS_SceneColorHDR/SCS_SceneColorHDRNoAlpha and TargetGamma as 2.2f

(2) Transparency material capturing looks not totally reflecting the art.
What would be the recommended settings of USceneCaptureComponent2D & UTextureRenderTarget2D to have a photorealistic captured output?

(3) There are some ghost effects in the captured images.

It would be great if you could help with some guidelines.

This is the expected image of transparency rendering, which is captured from the Standalone-run window:

Thanks a lot in advance!

[anonymous_user_b7505e2e]

Dear valued Unreal Engine user,

One option would be to take a capture of the scene using Render Doc or another tool specific to the graphics card. r.rhisetgpucaptureoptions 1 is a useful cvar to use for this.

You could then compare the passed in both the scene captured version and the standalone rendered version for differences. It may be that it parts are rendering correctly until it is performing the tone mapping, or that some specific show flags are not enabled.

Once you have narrowed down which bits are not working, you could then work out which settings need to change to match the standard rendering output.

Thank you for your continued support,

[Okaminu (Okami)]

Hi @SteelWoolAnthony,

So I then used Render Doc to capture the scene (with r.RHISetGPUCaptureOptions 1 as you suggested), which results in the same output as in the screenshot posted above (as the expected image of transparency rendering).
Having a look into the source code, I could see Render Doc invokes this function FRenderDocPluginModule::DoCaptureCurrentViewport():
https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Plugins/Developer/RenderDocPlugin/Source/RenderDocPlugin/Private/RenderDocPluginModule.cpp#L447
, which then basically calls FViewport::Draw().

This method has different capturing pipeline compared to the method I’m using USceneCaptureComponent2D::CaptureScene(), which issues an RHI ENQUEUE_RENDER_COMMAND(CaptureCommand) command then I read the pixel data back using ENQUEUE_RENDER_COMMAND(ReadSurfaceCommand)

So it seems the rendered contents returned by those are different, and is there any specific PostProcessSettings for Vulkan and Transparency capturing that I am supported to configure?

Thanks.

[Okaminu (Okami)]

Hi @ DanielW @TimHobson @Lovecraft_K @ddvlost
Would you happen to know about some info to help with this issue? Thank you.