[D3D12] ResourceBarrier State Mismatch Error in DispatchRays (Lumen + RDG)

Development Rendering
, ,
1

Hi everyone,

I am encountering a D3D12 Resource Barrier error when launching my project with -d3ddebug enabled. This issue seems to occur within the DispatchRays method when using Lumen with Hardware Ray Tracing enabled.

The Error: I receive the following error in the log:

LogD3D12RHI: Error: [D3DDebug] ID3D12CommandList::ResourceBarrier: Before state (0x2C0: D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT) of resource (0x0000015240C99240:'Lumen.Reflections.ClearUnusedTracingTileIndirectArgs') (subresource: 0) specified by transition barrier does not match with the state (0x240: D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE|D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT) specified in preceding ResourceBarrier or as InitialState

Investigation: I traced the issue to D3D12RayTracing.cpp inside the DispatchRays method:

CommandContext.AddBarrier(
    ArgumentBuffer->GetResource(),
    ED3D12Access::SRVGraphicsNonPixel | ED3D12Access::IndirectArgs,
    ED3D12Access::CopySrc,
    0);

Requested Transition: The code above requests ED3D12Access::SRVGraphicsNonPixel | ED3D12Access::IndirectArgs. In the underlying D3D12 implementation, this translates to 0x2C0 (D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT).

Actual State: The previous barrier (automatically collected by RDG) sets the state to IndirectArgs | SRVCompute. This translates to 0x240 (D3D12_RESOURCE_STATE_NON_PIXEL_SHADER_RESOURCE | D3D12_RESOURCE_STATE_INDIRECT_ARGUMENT).

The mismatch is clearly caused by the PIXEL_SHADER_RESOURCE flag being expected but not present in the previous state.

Question: I can temporarily resolve this by hardcoding the AccessBefore in CommandContext.AddBarrier to ED3D12Access::SRVCompute | ED3D12Access::IndirectArgs. However, this is brittle; if the upstream usage of ArgumentBuffer changes, this manual fix will break again.

The ArgumentBuffer is allocated via RDG (Render Graph), for example:

FRDGBufferRef HardwareRayTracingIndirectArgsBuffer = GraphBuilder.CreateBuffer(FRDGBufferDesc::CreateIndirectDesc<FRHIDispatchIndirectParameters>(1), TEXT("Lumen.RadianceCache.HardwareRayTracing.IndirectArgsBuffer"));
// ...
DispatchRayGenOrComputeShader(
    GraphBuilder,
    // ... args ...
    HardwareRayTracingIndirectArgsBuffer,
    // ... args ...
);

Since RDG handles resource pooling and aliasing, how can I correctly retrieve the actual previous state of the resource to manually set the Barrier in this low-level RHI context? Or is there a standard way to handle this transition within the RDG-to-RayTracing pipeline?

Reproduction Settings: I can reproduce this issue in the official UE 5.7.1 branch. The issue is primarily triggered when I disable Inline Ray Tracing and force RayGen Shaders (r.Lumen.HardwareRayTracing.LightingMode=2).

Here is my DefaultEngine.ini configuration:

;<Lumen>
r.Lumen.HardwareRayTracing=True
r.Lumen.ScreenProbeGather.ShortRangeAO=0
r.Lumen.DiffuseIndirect.SSAO=1
r.Lumen.TranslucencyReflections.Enable=1
r.Lumen.ScreenProbeGather.ScreenTraces.HZBTraversal.SkipHairHits=1
r.Lumen.Reflections.MaxRoughnessToTrace=0.2
r.Lumen.HardwareRayTracing.LightingMode=2
r.Lumen.Reflections.HardwareRayTracing.HitLightingMaxRoughness=0.1
r.Lumen.HardwareRayTracing.HitLighting.Skylight=1
r.Lumen.ScreenProbeGather.UseSHRadianceCache=1
r.Lumen.HardwareRayTracing.SurfaceCacheAlphaMasking=1
;<Lumen>

;<Ray Tracing>
r.RayTracing=True
r.RayTracing.Shadows=False
r.RayTracing.Skylight=False
r.RayTracing.Geometry.NiagaraMeshes=0
r.RayTracing.Geometry.NiagaraRibbons=0
r.RayTracing.Geometry.NiagaraSprites=0
r.RayTracing.Nanite.Mode=0
r.RayTracing.Nanite.StreamOut.MaxNumVertices=8388608
r.RayTracing.Nanite.StreamOut.MaxNumIndices=33554432
r.RayTracing.Nanite.MaxBuiltPrimitivesPerFrame=4194304
r.RayTracing.Nanite.MaxStagingBufferSizeMB=512
r.RayTracing.Nanite.BLASScratchSizeMultipleMB=32
r.RayTracing.PersistentSBT=0
;<Ray Tracing>

;<Async Compute>
r.D3D12.AllowAsyncCompute=1
r.RDG.AsyncCompute=1
r.RDG.AsyncComputeTransientAliasing=1
r.SkyAtmosphereAsyncCompute=1
r.Nanite.AsyncRasterization=1
r.Nanite.AsyncRasterization.CustomPass=1
r.Nanite.AsyncRasterization.LumenMeshCards=1
r.Nanite.AsyncRasterization.ShadowDepths=1
r.LocalFogVolume.TileCullingUseAsync=1
r.SceneDepthHZBAsyncCompute=0
r.Forward.LightGridAsyncCompute=1
r.RayTracing.AsyncBuild=1
r.Lumen.AsyncCompute=1
r.Lumen.DiffuseIndirect.AsyncCompute=1
r.Lumen.Reflections.AsyncCompute=0
r.LumenScene.Lighting.AsyncCompute=1
r.VolumetricRenderTarget.PreferAsyncCompute=1
r.TSR.AsyncCompute=1
;<Async Compute>

Any insights or suggestions would be greatly appreciated. Thanks!

2

Update: I think I found the solution.

It seems that RDG resource transitions should rely on RDG_BUFFER_ACCESS. For example:

class FLumenRadianceCacheHardwareRayTracing : public FLumenHardwareRayTracingShaderBase
{
    DECLARE_LUMEN_RAYTRACING_SHADER(FLumenRadianceCacheHardwareRayTracing)

    // Parameters
    BEGIN_SHADER_PARAMETER_STRUCT(FParameters, )
       RDG_BUFFER_ACCESS(HardwareRayTracingIndirectArgs, ERHIAccess::IndirectArgs | ERHIAccess::SRVCompute)
    // ...

I suspect the fix is to simply add an AccessBefore parameter to DispatchRays, which can then be passed in by DispatchRayGenOrComputeShader to ensure the correct state transition is used.