[D3D12] ResourceBarrier State Mismatch Error in DispatchRays (Lumen + RDG)

Development Rendering
, ,
1

Hi everyone,

I am encountering a D3D12 Resource Barrier error when launching my project with -d3ddebug enabled. This issue seems to occur within the DispatchRays method when using Lumen with Hardware Ray Tracing enabled.

The Error: I receive the following error in the log:

LogD3D12RHI: Error: [D3DDebug] ID3D12CommandList::ResourceBarrier: Before state (0x2C0: D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT) of resource (0x0000015240C99240:'Lumen.Reflections.ClearUnusedTracingTileIndirectArgs') (subresource: 0) specified by transition barrier does not match with the state (0x240: D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT) specified in preceding ResourceBarrier or as InitialState

Investigation: I traced the issue to D3D12RayTracing.cpp inside the DispatchRays method:

CommandContext.AddBarrier(
    ArgumentBuffer->GetResource(),
    ED3D12Access::SRVGraphicsNonPixel | ED3D12Access::IndirectArgs,
    ED3D12Access::CopySrc,
    0);

Requested Transition: The code above requests ED3D12Access::SRVGraphicsNonPixel | ED3D12Access::IndirectArgs. In the underlying D3D12 implementation, this translates to 0x2C0 (D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT).

Actual State: The previous barrier (automatically collected by RDG) sets the state to IndirectArgs | SRVCompute. This translates to 0x240 (D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT).

The mismatch is clearly caused by the PIXEL_SHADER_RESOURCE flag being expected but not present in the previous state.

Question: I can temporarily resolve this by hardcoding the AccessBefore in CommandContext.AddBarrier to ED3D12Access::SRVCompute | ED3D12Access::IndirectArgs. However, this is brittle; if the upstream usage of ArgumentBuffer changes, this manual fix will break again.

The ArgumentBuffer is allocated via RDG (Render Graph), for example:

FRDGBufferRef HardwareRayTracingIndirectArgsBuffer = GraphBuilder.CreateBuffer(FRDGBufferDesc::CreateIndirectDesc<FRHIDispatchIndirectParameters>(1), TEXT("Lumen.RadianceCache.HardwareRayTracing.IndirectArgsBuffer"));
// ...
DispatchRayGenOrComputeShader(
    GraphBuilder,
    // ... args ...
    HardwareRayTracingIndirectArgsBuffer,
    // ... args ...
);

Since RDG handles resource pooling and aliasing, how can I correctly retrieve the actual previous state of the resource to manually set the Barrier in this low-level RHI context? Or is there a standard way to handle this transition within the RDG-to-RayTracing pipeline?

Reproduction Settings: I can reproduce this issue in the official UE 5.7.1 branch. The issue is primarily triggered when I disable Inline Ray Tracing and force RayGen Shaders (r.Lumen.HardwareRayTracing.LightingMode=2).

Here is my DefaultEngine.ini configuration:

;<Lumen>
r.Lumen.HardwareRayTracing=True
r.Lumen.ScreenProbeGather.ShortRangeAO=0
r.Lumen.DiffuseIndirect.SSAO=1
r.Lumen.TranslucencyReflections.Enable=1
r.Lumen.ScreenProbeGather.ScreenTraces.HZBTraversal.SkipHairHits=1
r.Lumen.Reflections.MaxRoughnessToTrace=0.2
r.Lumen.HardwareRayTracing.LightingMode=2
r.Lumen.Reflections.HardwareRayTracing.HitLightingMaxRoughness=0.1
r.Lumen.HardwareRayTracing.HitLighting.Skylight=1
r.Lumen.ScreenProbeGather.UseSHRadianceCache=1
r.Lumen.HardwareRayTracing.SurfaceCacheAlphaMasking=1
;<Lumen>

;<Ray Tracing>
r.RayTracing=True
r.RayTracing.Shadows=False
r.RayTracing.Skylight=False
r.RayTracing.Geometry.NiagaraMeshes=0
r.RayTracing.Geometry.NiagaraRibbons=0
r.RayTracing.Geometry.NiagaraSprites=0
r.RayTracing.Nanite.Mode=0
r.RayTracing.Nanite.StreamOut.MaxNumVertices=8388608
r.RayTracing.Nanite.StreamOut.MaxNumIndices=33554432
r.RayTracing.Nanite.MaxBuiltPrimitivesPerFrame=4194304
r.RayTracing.Nanite.MaxStagingBufferSizeMB=512
r.RayTracing.Nanite.BLASScratchSizeMultipleMB=32
r.RayTracing.PersistentSBT=0
;<Ray Tracing>

;<Async Compute>
r.D3D12.AllowAsyncCompute=1
r.RDG.AsyncCompute=1
r.RDG.AsyncComputeTransientAliasing=1
r.SkyAtmosphereAsyncCompute=1
r.Nanite.AsyncRasterization=1
r.Nanite.AsyncRasterization.CustomPass=1
r.Nanite.AsyncRasterization.LumenMeshCards=1
r.Nanite.AsyncRasterization.ShadowDepths=1
r.LocalFogVolume.TileCullingUseAsync=1
r.SceneDepthHZBAsyncCompute=0
r.Forward.LightGridAsyncCompute=1
r.RayTracing.AsyncBuild=1
r.Lumen.AsyncCompute=1
r.Lumen.DiffuseIndirect.AsyncCompute=1
r.Lumen.Reflections.AsyncCompute=0
r.LumenScene.Lighting.AsyncCompute=1
r.VolumetricRenderTarget.PreferAsyncCompute=1
r.TSR.AsyncCompute=1
;<Async Compute>

Any insights or suggestions would be greatly appreciated. Thanks!

2

Update: I think I found the solution.

It seems that RDG resource transitions should rely on RDG_BUFFER_ACCESS. For example:

class FLumenRadianceCacheHardwareRayTracing : public FLumenHardwareRayTracingShaderBase
{
    DECLARE_LUMEN_RAYTRACING_SHADER(FLumenRadianceCacheHardwareRayTracing)

    // Parameters
    BEGIN_SHADER_PARAMETER_STRUCT(FParameters, )
       RDG_BUFFER_ACCESS(HardwareRayTracingIndirectArgs, ERHIAccess::IndirectArgs | ERHIAccess::SRVCompute)
    // ...

I suspect the fix is to simply add an AccessBefore parameter to DispatchRays, which can then be passed in by DispatchRayGenOrComputeShader to ensure the correct state transition is used.

3

I was hitting this too in 5.7.4 while trying to use -d3ddebug, and after debugging, the issue stems from this seemingly innocuous line in GetD3D12ResourceState in D3D12LegacyBarriers.cpp:

// Translate the requested after state to a D3D state
if (EnumHasAnyFlags(D3D12AccessWithoutDiscard, ED3D12Access::SRVGraphics) && InQueueType == ED3D12QueueType::Direct)
{
	State |= D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | ExtraReadState;
}

SRVGraphics is 2 flags OR’d together (SRVGraphicsPixel | SRVGraphicsNonPixel), which means if either flag is true, EnumHasAnyFlags returns true. In this Lumen case, because SRVGraphicsNonPixel is set, the if is evaluated to true and it OR’s in the wrong flags into State. Splitting the if up into 2 if’s that check SRVGraphicsPixel and SRVGraphicsNonPixel separately and then ORing in the appropriate D3D12 flag fixes the issue.

// Translate the requested after state to a D3D state
if (EnumHasAnyFlags(D3D12AccessWithoutDiscard, ED3D12Access::SRVGraphicsPixel) && InQueueType == ED3D12QueueType::Direct)
{
	State |= D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE | ExtraReadState;
}
if (EnumHasAnyFlags(D3D12AccessWithoutDiscard, ED3D12Access::SRVGraphicsNonPixel))
{
	State |= D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | ExtraReadState;
}

NVidia’s Streamline also seems to have a similar bug.

Now I’m hitting a lot of

Error: [D3DDebug] ID3D12CommandQueue1::ExecuteCommandLists: Placed resources, reserved resources, or committed resources with D3D12_HEAP_FLAG_CREATE_NOT_ZEROED flag with either render target or depth stencil flags must be initialized with a Discard/Clear/Copy operations before other operations are supported

though :slight_smile: , so I’ll have to dig into this.