Crash when switching to Wireframe (F1) with FMeshPassProcessor

Hello,

We are working on a custom mesh rendering pass for our game.

It is based on a FMeshPassProcessor without any engine modification (game module).

Meshes are registered in a world subsystem, with their bRenderInMainPass property disabled. They are not using Nanite. A scene view extension handles creating the FMeshPassProcessor instance (in PostRenderBasePassDeferred_RenderThread), looping over all registered mesh proxies within a AddSimpleMeshPass call (see code at the bottom).

When in-game (PIE), and switching to Wireframe view with F1, an ensure is raised (FRHIPassInfo::Validate) :

// Ensure NumSamples matches with color RT
if (NumSamples != -1)
{
    ensure(DepthStencilRenderTarget.DepthStencilTarget->GetNumSamples() == NumSamples);
}

Here, NumSamples is 4 in Wireframe view while the D/S attachment only has 1 sample, triggering the condition. After the ensure, the GPU device hangs, crashing the game. As a side note, switching to Eject mode (F8), and then manually selecting the Wireframe viewmode from the dropdown doesn’t trigger this condition.aa

How could we make our FTransparencyMeshPassProcessor compatible with wireframe, or at least disable it in order to avoid crashing ?

We have already tried detecting the viewmode inside the SVE’s IsActiveThisFrame_Internal method, but the crash happens right at the frame when pressing F1, where the previous draw command is already in flight.

if (GIsEditor && GEditor)
{
    for (FLevelEditorViewportClient *const LevelVC : GEditor->GetLevelViewportClients())
    {
       if (LevelVC->Viewport == Context.Viewport)
       {
          if (!LevelVC->IsViewModeEnabled(EViewModeIndex::VMI_Lit))
          {
             return false;
          }
       }
    }
...

Thanks in advance,

Tim

---------------

Drawing code within the Scene View Extension’s PostRenderBasePassDeferred_RenderThread :

AddSimpleMeshPass(GraphBuilder, PassParameters, InView.Family->Scene->GetRenderScene(), InView, nullptr, RDG_EVENT_NAME("Transparency %dx%d - %u prims", RTSize.X, RTSize.Y, RegisteredPassPrimitives.Num()), static_cast<const FViewInfo&>(View).ViewRect,
		[&InView, this](FDynamicPassMeshDrawListContext* DynamicMeshPassContext)
		{
			FTransparencyMeshPassProcessor Processor(InView.Family->Scene->GetRenderScene(), &InView, DynamicMeshPassContext);
			Processor.SetOpacity(FMath::Sin(GFrameCounter * 0.25f) * 0.5f + 0.5f);
			for (const FPrimitiveSceneProxy *const Primitive : RegisteredPassPrimitives)
			{
				if (Primitive == nullptr)
				{
					return;
				}


				TArray<FMeshBatch> MeshElements;
				Primitive->GetMeshDescription(0, MeshElements);
				if (MeshElements.IsEmpty())
				{
					continue;
				}
				
				for (const FMeshBatch& Batch : MeshElements)
				{
					Processor.AddMeshBatch(Batch, ~0ull, Primitive);
				}
			}
		});

Hello there,

Thanks for reaching out. Based on your information, MSAA is enabled on the depth stencil render target used for the wireframe pass, so you will need to resolve it down to one sample before that, or as a simple fix, deactivate it. How are you creating your render targets for your mesh passes, and how are they being passed from your custom code to the wireframe pass? I assume that is in your custom FTransparencyMeshPassProcessor, which I cannot find in our codebase. Try to track down where the depth stencil attachments are being created, and as a simple first solution, merely disable MSAA and see if that gets rid of your issue. Please let me know if you have any further questions about this.

Hi again,

Yeah, that does seem strange to me. However, you seem to have access to the SceneDepthTexture on line 6. Have you tried just updating the descriptor to make sure it takes four samples? Otherwise, if you can share the code with me in a small repro project, then I could also take a closer look from my end. Let me know what you think!

Thanks, Tim, for putting together that repro project. I am reproducing this crash now, but I will need more time to investigate it further. This will likely happen after our company’s summer break, which will end by mid-July. Let me know if you need more urgent help with this, and I can escalate it before all of this is off after tomorrow.

Hi again, Tim. Sorry for the long delay. After the break, I updated my 5.5 engine build to our latest CL, and I noticed that I am not crashing in the project anymore when I enter PIE, although the wireframe is now not matching the view that I would expect (see screenshot). Can you tell me what CL your engine is currently at, and I can see if there are differences between your version and mine? Thanks.

[Image Removed]

Hello,

That’s weird ! The repro project has been created using the latest vanilla release 5.5.4 (Perforce changelist 40574608). In our project we are using release 5.5.1.

If that’s the case, then there is probably something going wrong on my end. My apologies for not responding sooner. I got caught up with some urgent tasks in the last few days, and I have not gotten around to looking at this further. I will update you as soon as I have more, though. Thanks for your patience for now, though.

Hi Tim,

I had some more time to work on your issue, but I have unfortunately reached a dead end with it. Without making engine modifications, I could not find a way to transition the depth/stencil attachment to also be multisampled at the time your scene view extension is called. I have filed an issue with the developer team, who will look closely at this as soon as possible. You can track the progress of the Jira here: Unreal Engine Issues and Bug Tracker (UE\-312676\). I am sorry for not being able to find a proper solution at this time. If you have any more questions, please feel free to contact me.

Hello again,

So I have gotten word from our devs that the crash you are running into might have been fixed in 5.6 already. Can you upgrade your project to 5.6 to see if you are still running into this crash? Please let me know what you think.

Hi Tim,

Thanks for letting me know, and I am sorry to hear the bug has not been fixed yet. Could you please send me the upgraded version (5.6.1) of your project? That way, I can add it to the ticket and have the devs take another look.

Thanks, Tim, for the quick turnaround. I have updated the Jira with the new project. The team will get to this ticket soon and find a solution for the crash. I will close the ticket for now, but please contact me again if you need anything else.

Hello Tim,

I am re-opening to let you know that I have an update for your ticket that needs your follow-up. We’d like to know why you’re trying to change the depth stencil render target, since the wireframe mode has different targets and will not be compatible. If you comment the following lines in your sample project, the project runs without crashing.

PassParameters->RenderTargets = RenderTargets;
PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(SceneTextures->GetContents()->SceneDepthTexture, ERenderTargetLoadAction::ELoad, ERenderTargetLoadAction::ELoad, FExclusiveDepthStencil::DepthWrite_StencilWrite);

Cheers,

Tim

Hi Tim,

Sorry for the confusion. I told you to comment on two lines in your code, where only one should have been. The second line should only be removed:

PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(SceneTextures->GetContents()->SceneDepthTexture, ERenderTargetLoadAction::ELoad, ERenderTargetLoadAction::ELoad, FExclusiveDepthStencil::DepthWrite_StencilWrite);This should allow you to keep the render targets bound for the other view modes. Otherwise, if that is not an acceptable solution, you can use the workaround you created.

Great, I am glad that we have been able to find an acceptable compromise! I will consider this ticket closed then, but if you have any follow-up questions, please contact us again.

Hello,

Thanks for your answer. MSAA seems to be enabled by default for wireframe (F1) pass, we don’t really want to disable it just to fix that issue.

My custom FMeshPassProcessor (named FTransparencyMeshPassProcessor) doesn’t create any render target or depth/stencil attachment.

My SceneViewExtension does a AddSimpleMeshPass call, with the following PassParameters

void FTransparencySceneViewExtension::PostRenderBasePassDeferred_RenderThread(FRDGBuilder& GraphBuilder, FSceneView& InView,
	const FRenderTargetBindingSlots& RenderTargets, TRDGUniformBufferRef<FSceneTextureUniformParameters> SceneTextures)
{
    // [0] : SV_Target0, ... [7] : SV_Target7
    PassParameters->RenderTargets = RenderTargets;
    PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(SceneTextures->GetContents()->SceneDepthTexture, ERenderTargetLoadAction::ELoad, ERenderTargetLoadAction::ELoad, FExclusiveDepthStencil::DepthWrite_StencilWrite);
...
   AddSimpleMeshPass(GraphBuilder, PassParameters, InView.Family->Scene->GetRenderScene(), InView, nullptr, RDG_EVENT_NAME("Transparency %dx%d - %u prims", RTSize.X, RTSize.Y, RegisteredPassPrimitives.IsValid() ? RegisteredPassPrimitives->Num() : 0u), static_cast<const FViewInfo&>(View).ViewRect,
		[&InView, this](FDynamicPassMeshDrawListContext* DynamicMeshPassContext)
		{
                     // Rendering code, creates and uses our FTransparencyMeshPassProcessor
                });
}

This effectively allows to draw on the active render targets. I don’t have to resolve render targets there.

It just seems that for a single frame, when wireframe is turned on (F1), the color MRT are set to the correct number of samples (4) but not the associated dpeth/stencil attachment (all being created by UE’s renderer), which have 1 sample.

Since the previous AddSimpleMeshPass pass is already in flight, it will fail during the Validate step (before the lambda is called).

Because we’re only rendering to existing render targets, passed to the SceneViewExtension, and not allocating/creating them, I believe this is an Unreal Engine bug.

Feel free to ask me for more details about our implementation !

Have a great day,

Tim

Hey !

Unfortunately, the descriptor of the FRDGTexture (for the depth/stencil attachment) is const at that point, the number of samples can not be changed.

I made an independent repro project for UE 5.5.4 (exported as a .zip file).

My code has been moved to a C++ plugin, and I created a map with a single blueprint that registers a static mesh to be processed by my FMeshPassProcessor.

You can start the PIE, and hit F1 to go into Wireframe mode and observe that the engine instantly crashes.

Thanks for your time and dedication !

Hello, thank you for having a look at the project ! I’m glad you could reproduce the crash on your side. Don’t worry this is not urgent matter, enjoy your holidays !

Hello,

Sorry for the delay, I just got back from vacation.

I downloaded UE 5.6.1 from the Epic Games Launcher, upgraded the repro project, and started a PIE.

The crash is still happening, in the same conditions. Maybe the devs refer to a certain CL that hasn’t been merged in 5.6.1 yet, fixing the issue ?

Otherwise, we’ll settle on making extra checks to disable this rendering feature entirely when in wireframe mode in editor.

Thanks for your dedication !

Hello,

Here’s the 5.6.1 converted project, I hope that will help !

Hello,

Commenting these two lines indeed prevents the crash in wireframe mode but makes the whole render pass output nothing in other modes too. That’s because there are no render targets bound anymore.

This is not a viable solution for us.

What we will implement is a simple test :

  • Is there a number of sample > 1 set up on at least one RenderTarget (or depth/stencil attachment)
  • If that’s the case, we know we are in wireframe, we can disable the pass altogether

Here’s the code I came up with (to insert in TransparencySceneViewExtension::PostRenderBasePassDeferred_RenderThread

) :

if (FRDGTexture *DepthStencilTexture{RenderTargets.DepthStencil.GetTexture()})
	{
		const uint8 DepthStencilTextureSamples{DepthStencilTexture->Desc.NumSamples};
		for (int32 RTIndex{0}; RTIndex < RenderTargets.Output.Num(); ++RTIndex)
		{
			if (FRDGTexture *Texture{RenderTargets[RTIndex].GetTexture()})
			{
				if (Texture->Desc.NumSamples != DepthStencilTextureSamples || Texture->Desc.IsMultisample())
				{
					// Wireframe mode detected
					return;
				}
			}
		}
	}