SceneCapture that running on mainRenderer make skeletalMesh LOD flickering

Development Rendering
, ,
1

[investigation]

When SceneCapture2D uses the main renderer, it gets added to the CustomRenderPass based on what it captures.

// SceneCaptureRendering.cpp
// Func : FScene::UpdateSceneCaptureContents(USceneCaptureComponent2D* CaptureComponent)
// As optimization for depth capture modes, render scene capture as additional render passes inside the main renderer.
if (GSceneCaptureAllowRenderInMainRenderer && 
    CaptureComponent->bRenderInMainRenderer && 
    (CaptureComponent->CaptureSource == ESceneCaptureSource::SCS_SceneDepth || CaptureComponent->CaptureSource == ESceneCaptureSource::SCS_DeviceDepth ||
     CaptureComponent->CaptureSource == ESceneCaptureSource::SCS_BaseColor || CaptureComponent->CaptureSource == ESceneCaptureSource::SCS_Normal)
    )
{
// ... other things...
AddCustomRenderPass(nullptr, PassInput);
}

In AllViews, the FViewInfo from CustomRenderPassInfos is added after the original FViewInfo of the main renderer.

// SceneRendering.cpp
// Func : FSceneRenderer::FSceneRenderer(const FSceneViewFamily* InViewFamily, FHitProxyConsumer* HitProxyConsumer)
AllViews.Empty(Views.Num() + NumAdditionalViews);
for (int32 i = 0; i < Views.Num(); ++i)
{
    AllViews.Add(&Views[i]);
}
for (FCustomRenderPassInfo& PassInfo : CustomRenderPassInfos)
{
    for (FViewInfo& View : PassInfo.Views)
    {
       AllViews.Add(&View);
    }
}

The visibility tasks are processed sequentially using a for loop, so the viewInfo added by CustomRenderPass is processed afterwards. As a result, the ViewInfo that comes later ends up being used as the final one.

// SceneVisibility.cpp
// Func : void FVisibilityTaskData::LaunchVisibilityTasks(const UE::Tasks::FTask& BeginInitVisibilityPrerequisites)
for (int32 ViewIndex = 0; ViewIndex < Views.Num(); ++ViewIndex)
{
    // Each view gets its own visibility task packet which contains all the state to manage the task graph for a view.
    FVisibilityViewPacket& ViewPacket = ViewPackets.Emplace_GetRef(*this, Scene, *Views[ViewIndex], ViewIndex);
    ...
}

The View information is passed to SkeletalMeshSceneProxy and used for LOD calculation.

// SkeletalMesh.cpp
FPrimitiveViewRelevance FSkeletalMeshSceneProxy::GetViewRelevance(const FSceneView* View) const

I want to use MainRenderer for SceneCapture due to optimization concerns.

To solve this issue, Iā€™m trying to add a flag member to the FSceneView structure that determines whether it should be used for LOD calculations.

I found a value bUseFieldOfViewForLOD in FViewInfo, but it doesnā€™t seem to be used anywhere.

I would appreciate any advice regarding this issue.

Thanks.

2

ģž¬ķ˜„ ė°©ė²•

  1. Place a SceneCapture2D and use option `Render In Main Renderer`.
  2. Set option `BaseColor` in `RGB for Capture Source`.
  3. Make sure the SceneCapture2D capture skeletal mesh kind of characters.
  4. Run the game, check Mesh LOD coloration and youā€™ll notice flickering.

Iā€™ve attached a reproducible project so you can just run game and check LODs.

I use 5.5.4 launch version for pasted project.

3

Weā€™ve identified the issue, and have a fix. There are two parts to the fix, if you want to build the engine locally to apply it. The first is to FSceneRenderer::FSceneRenderer in Engine/Source/Runtime/Renderer/Private/SceneRendering.cpp, where the following frame values need to be initialized, on the line after CustomRenderPassInfo->ViewFamily.Time is initialized:

		CustomRenderPassInfo->ViewFamily.FrameNumber = ViewFamily.FrameNumber;
		CustomRenderPassInfo->ViewFamily.FrameCounter = ViewFamily.FrameCounter;

The second is to change this line in Engine/Source/Runtime/Engine/Private/SkeletalRender.cpp:

	// When rendering multiple views we need to guard the assignment with a mutex since relevance can occur in parallel.
	const bool bMultiView = View->Family->Views.Num() > 1;

To this, so it includes the Custom Render Pass views when deciding whether multi-view thread safety logic needs to run:

const bool bMultiView = View->Family->AllViews.Num() > 1;This code fix will be in 5.8, but the relevant code is the same in any older version of the engine which includes the ā€œRender In Main Rendererā€ flag, so you should be able to apply the fix to any version.

Sorry for the inconvenience!

--Jason Hoerner

4

I applied it and it seems to work well. Thanks.