It’s unclear to me how FSimulationShaderDefaultCS is supposed to compile with SM5 set for the project, is it intended for SM6 only? This is what I’m seeing locally, so it’s unclear to me what’s happening differently during the cook process and consequently what’s going on in the runtime. But the end result is the runtime is encountering crashes, which once we managed to get lucky and have an internal repro with aftermath enabled, indicated FSimulationShaderDefaultCS as the culprit.
Steps to Reproduce
We are seeing a high occurrence of GPU crashes due to an out of bounds memory access in FSimulationShaderDefaultCS. While attempting to get a local binary shader file for NSight, we’ve found that the shader doesn’t compile.
To reproduce the failed compilation:
Choose SM5 as the shader model for the project.
In the editor, run r.ShaderDevelopmentMode 1 and then recompileshaders NiagaraStatelessSimulationDefault.usf
This generates a Saved\ShaderDebugInfo\PCD3D_SM5\Global\FSimulationShaderDefaultCS\0 folder, with DebugCompile.in, DebugCompileArgs.txt NiagaraStatelessSimulationDefault.usf, OutputHash.txt
Run <root>\Engine\Binaries\Win64\ShaderCompileWorker.exe “<root>/Projects/TFT/Saved/ShaderDebugInfo/PCD3D_SM5/Global/FSimulationShaderDefaultCS/0” 0 “DebugCompile” DebugCompile.in DebugCompile.out -DebugSourceFiles=“NiagaraStatelessSimulationDefault.usf” -TimeToLive=0.0f -KeepInput
This generates CompileFXC.bat, NiagaraStatelessSimulationDefault.intermediate.spvasm, NiagaraStatelessSimulationDefault.intermediate.hlsl, Output.d3dasm
Modify CompileFXC.bat to point to an actual fxc.exe location
It’s likely adding a BRANCH to the helpers will fix the issue. We verified there were no initial performance regressions with the change but haven’t verified it fixes the issue yet.
The reason CompileFXC.bat doesn’t work is because it outputs the compile commands only for the most optimistic version - the one where FXC.exe works the first time because that debug bat file is generated early on in the function. CompileAndProcessD3DShaderFXCExt in D3DShaderCompiler.cpp actually has fallbacks if FXC.exe fails and it doesn’t always output to the log when the first compile attempt fails if the fallback succeeds. In the case of this shader it looks like it fell back to the DXC → SPIR-V → SPIRV-Cross HLSL 5.0 → FXC path and likely you can produce the final DXBC by changing the CompileFXC.bat so it uses the NiagaraStatelessSimulationDefault.intermediate.hlsl as input.