In the end I was able to make it work under Windows and DX12, moving from using FRHIRayTracingScene, now deprecated and probably broken, to FRHIShaderBindingTable
Basically:
FRHIShaderBindingTable* SBT = RayTracingSBT->AllocateRHI(RHICmdList, ERayTracingShaderBindingMode::RTPSO, ERayTracingHitGroupIndexingMode::Allow, 1, 0, PSOInitializer.GetMaxLocalBindingDataSize());
and use it with RHICmdLis
RHICmdList.SetRayTracingMissShader(SBT, 0, PipeLine, 0 /* ShaderIndexInPipeline */, 0, nullptr, 0);
RHICmdList.CommitShaderBindingTable(SBT);
RHICmdList.RayTraceDispatch(
PipeLine,
GetGlobalShaderMap(GMaxRHIFeatureLevel)->GetShader<FRayGenTestRGS>().GetRayTracingShader(),
SBT,
GlobalResources,
TextureSize.X,
TextureSize.Y
);
Unfortunately using Vulkan the same code does not work, it hungs for a while and then I get VK_Device_Lost error whiel retrieving the FRDGBufferSRVRef layerView
So it’s still WIP