RHI causes Errors upgrading from UE 5.0 to 5.1 on NVIDIA's Dataset_Synthesizer Plugin

Development Programming & Scripting C++
, , , ,
1

So I forked this UE 4.27.2 version of NVIDIA’s Dataset_Synthesizer Plugin (GitHub - saratrajput/Dataset_Synthesizer at develop/UE4.27.2). And I need to upgrade it to UE 5.2.1 for my work at my university.
It basically generates labeled data for Image recognition AI’s, by capturing a normal image of the Scene, a segmentated image, a depth image and a json for every frame.

When uprading to newer Versions there were several warnings and errors, that were easy to fix - and I got it running on UE 5.0.3 - but for UE 5.1.1, after successfully compiling, I get these errors:

//

  • Crash with NumSamples = 1 (No Change):

Assertion failed: !ResolveRT || ResolveRT->IsMultisampled() [File:C:\Programs\Epic Games\UE_5.1\Engine\Source\Runtime\RHI\Public\RHIResources.h] [Line: 3792]

UnrealEditor_NVSceneCapturer!FNVTextureReader::CopyTexture2d() [C:\Users\burak\Documents\Unreal Projects\Dataset_Synthesizer_5_1 - After Fix\Source\Plugins\NVSceneCapturer\Source\NVSceneCapturer\Private\NVTextureReader.cpp:279]
UnrealEditor_NVSceneCapturer!<lambda_3717573479555e0342a37edddbf3e7eb>::operator()() [C:\Users\burak\Documents\Unreal Projects\Dataset_Synthesizer_5_1 - After Fix\Source\Plugins\NVSceneCapturer\Source\NVSceneCapturer\Private\NVTextureReader.cpp:226]
UnrealEditor_NVSceneCapturer!TEnqueueUniqueRenderCommandType<FNVTextureReader::ReadPixelsRaw'::14’::ReadPixelsFromTextureName,<lambda_3717573479555e0342a37edddbf3e7eb> >::DoTask() [C:\Programs\Epic Games\UE_5.1\Engine\Source\Runtime\RenderCore\Public\RenderingThread.h:206]
UnrealEditor_NVSceneCapturer!TGraphTask<TEnqueueUniqueRenderCommandType<FNVTextureReader::ReadPixelsRaw'::14’::ReadPixelsFromTextureName,<lambda_3717573479555e0342a37edddbf3e7eb> > >::ExecuteTask() [C:\Programs\Epic Games\UE_5.1\Engine\Source\Runtime\Core\Public\Async\TaskGraphInterfaces.h:1348]
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_RenderCore
UnrealEditor_RenderCore
UnrealEditor_Core
UnrealEditor_Core
kernel32
ntdll

//

So i thought, ok lets just Multisample then, since it somehow is now a requirement and reason enough to crash the engine. This came out:

  • Crash with NumSamples = 2 (changed to 2):
    Error
    Out of video memory trying to allocate a rendering resource. Make sure your video card has the minimum required memory, try lowering the resolution and/or closing other applications that are running. Exiting…
    ->OK
    → Engine closes

(I don’t understand how this can be the case since I have a RTX 3060 with 12 GB Dedicated GPU memory, 16GB Shared GPU memory and 32GB of RAM, and all i am trying to do is render a single multisampled image)

//

Since all this worked in UE 5.0.3 I thought, okay maybe I just need to access one of the alternative functions (see code below), which do not require multisampling. But I need that optional resolve, which only the error causing function is providing. so I used the next closest function and changed the other variables, the new function does not change, to what it was like in UE 5.0.3. But then I get basically the same error deeper down the line in RHICore.cpp:

  • Crash with NumSamples = 1 and the workaround:
    Assertion failed: SourceDesc.IsMultisample() && !DestDesc.IsMultisample() [File:D:\build++UE5\Sync\Engine\Source\Runtime\RHICore\Private\RHICore.cpp] [Line: 28]

UnrealEditor_RHICore
UnrealEditor_D3D12RHI
UnrealEditor_RHI
UnrealEditor_RHI
UnrealEditor_RHI
UnrealEditor_RHI
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_RenderCore
UnrealEditor_Core
UnrealEditor_Core
kernel32
ntdll

Error causing code:

NVTextureReader.cpp (it basically looks like this: Dataset_Synthesizer/Source/Plugins/NVSceneCapturer/Source/NVSceneCapturer/Private/NVTextureReader.cpp at develop/UE4.27.2 · saratrajput/Dataset_Synthesizer (github.com))

//inside bool FNVTextureReader::ReadPixelsRaw(...) 

                FRHITextureCreateDesc TextureDesc;
                TextureDesc.SetExtent(TargetSize.X, TargetSize.Y)
                           .SetFormat(TargetPixelFormat)
                           .SetNumMips(1)
                           .SetNumSamples(1) // or .SetNumSamples(2)
                           .SetFlags(TexCreate_CPUReadback);

                FTexture2DRHIRef ReadbackTexture = RHICreateTexture(TextureDesc);

                CopyTexture2d(RendererModule, RHICmdList, NewSourceTexture, SourceRect, ReadbackTexture, FIntRect(FIntPoint::ZeroValue, TargetSize), bOverwriteAlpha);


//inside void FNVTextureReader::CopyTexture2d
        FRHITexture* DestRenderTargetTexture = ResampleTexturePooledRenderTarget->GetRHI();
        
        //Before:
        // FRHIRenderPassInfo RPInfo(DestRenderTarget.TargetableTexture, ERenderTargetActions::Load_Store, ReadbackTexture);

        // 5.1 adaptation
        FRHIRenderPassInfo RPInfo(DestRenderTargetTexture, ERenderTargetActions::Load_Store, ReadbackTexture);

        // isMultisampled() Workaround
        // FRHITexture* ColorRTs[] = {DestRenderTargetTexture};
        // FRHIRenderPassInfo RPInfo(1, ColorRTs, ERenderTargetActions::Load_Store);
        // RPInfo.ColorRenderTargets[0].ResolveTarget = ReadbackTexture;
        // RPInfo.ColorRenderTargets[0].MipIndex = 0;

RHIResources.h ( UnrealEngine/Engine/Source/Runtime/RHI/Public/RHIResources.h at 5.0 · EpicGames/UnrealEngine (github.com))

// UE 5.0 Version
	// Color, no depth, optional resolve, optional mip, optional array slice
	explicit FRHIRenderPassInfo(FRHITexture* ColorRT, ERenderTargetActions ColorAction, FRHITexture* ResolveRT = nullptr, uint8 InMipIndex = 0, int32 InArraySlice = -1)
	{
		check(ColorRT);
		ColorRenderTargets[0].RenderTarget = ColorRT;
		ColorRenderTargets[0].ResolveTarget = ResolveRT;
		ColorRenderTargets[0].ArraySlice = InArraySlice;
		ColorRenderTargets[0].MipIndex = InMipIndex;
		ColorRenderTargets[0].Action = ColorAction;
		DepthStencilRenderTarget.DepthStencilTarget = nullptr;
		DepthStencilRenderTarget.Action = EDepthStencilTargetActions::DontLoad_DontStore;
		DepthStencilRenderTarget.ExclusiveDepthStencil = FExclusiveDepthStencil::DepthNop_StencilNop;
		DepthStencilRenderTarget.ResolveTarget = nullptr;
		bIsMSAA = ColorRT->GetNumSamples() > 1;
		FMemory::Memzero(&ColorRenderTargets[1], sizeof(FColorEntry) * (MaxSimultaneousRenderTargets - 1));
	}

// UE 5.1 Version
	// Color, no depth, optional resolve, optional mip, optional array slice
	explicit FRHIRenderPassInfo(FRHITexture* ColorRT, ERenderTargetActions ColorAction, FRHITexture* ResolveRT = nullptr, uint8 InMipIndex = 0, int32 InArraySlice = -1)
	{
		check(!ResolveRT || ResolveRT->IsMultisampled()); // Crash causing line
		check(ColorRT);
		ColorRenderTargets[0].RenderTarget = ColorRT;
		ColorRenderTargets[0].ResolveTarget = ResolveRT;
		ColorRenderTargets[0].ArraySlice = InArraySlice;
		ColorRenderTargets[0].MipIndex = InMipIndex;
		ColorRenderTargets[0].Action = ColorAction;
	}

I have worked countless hours on this by now, and I now have run out of options I can think of. I hope that someone who understands this stuff better than I do can find a solution to this.

1 Like