Crashes in ShaderCompileWorker.exe (dxcompiler.dll) when using Incredibuild and cooking shaders for the first time in 5.7 projects

Development Pipeline & Plugins
, ,
1

Whilst working on upgrading a project to 5.7, we noticed errors like the following when cooking for Win64 for the first time:

LogShaderCompilers: Error: Shader compiler returned a non-zero error code (-1073741819).

The errors disappeared in subsequent cooks, which suggested that despite the error above the shaders had made their way to the DDC. In fact, if using -noshaderddc we are then able to reproduce these problems on every cook, although the number of errors the cooks hit seems to vary on each run. The errors always occurred when compiling shaders for PCD3D_SM6 (not with other shader formats) and early in the cook process.

The error code above maps to 0xC0000005 (“The instruction at … referenced memory at … The memory could not be …”). The bespoke exception handler in ShaderCompileWorker didn’t seem to trigger as no crash output files were being generated.

Something to note is that we are using Incredibuild. We tried invoking ShaderCompileWorker directly and also using UBA instead, and interestingly the errors did not seem to appear.

I should also mention that this problem is not specific to the project in question as we have been able to reproduce the same crashes on stock 5.7.1 using QAGame.

Eventually we managed to attach a debugger to crashing ShaderCompileWorker instances and got the callstack attached to the question and the exception details: “Exception thrown at 0x00007FFC8D8C2A95 (ntdll.dll) in ShaderCompileWorker.exe: 0xC0000005: Access violation reading location 0x0000000000000000.”

The crash occurring after main() explains that the bespoke exception handler in ShaderCompileWorker was not having an effect, and could explain why the shaders made their way to the DDC (as the ShaderCompileWorker would have generated all the output files prior to the crash), but we did not confirm this last detail.

We then decided to replace the dxcompiler.dll in 5.7 with the version that was included in 5.6, and the problems went away. This suggested an incompatibility between 5.7’s dxcompiler.dll and either Incredibuild or ShaderCompileWorker’s handling of Incredibuild.

We then noticed that ShaderCompileWorker uses module bootstrapping when run under Incredibuild. We have run some tests and so far the errors do not seem to occur when preventing all ShaderCompileWorkers from bootstrapping. When the crashes do occur, so far we have always seen warnings indicating a worker could not load the bootstrapping file, which according to a comment in ShaderCompileWorker.cpp is expected behaviour due to an issue in XGE, though this may suggest that changes in the way modules are loaded and unloaded by the different workers may be a factor in the crash.

We are not blocked by this issue at present as using 5.6’s dxcompiler.dll works for us for now (Perforce history suggested nothing in the new version was a must-have) and it could also be argued that we could add a workaround to ignore these particular errors given the crash happens after ShaderCompileWorker has successfully generated its outputs but in the long term it would be preferable if a proper resolution is found.

[Attachment Removed]

2

Steps to Reproduce

  • Ensure Incredibuild is installed (and check it’s used to compile shaders)
  • Ensure a given project targets the shader format PCD3D_SM6
  • Cook the project for the first time or pass -noshaderddc when cooking
  • Observe a number of occurrences of the error “LogShaderCompilers: Error: Shader compiler returned a non-zero error code (-1073741819).” early in the cook process (when compiling global shaders). They indicate a crash in ShaderCompileWorker.exe (details in the description).
    [Attachment Removed]
3

We also ran into this crash, and you just narrowly beat us to reporting it. : )

Our workaround was also to roll back to the UE5.6 version of dxcompiler.dll - however, we got a slightly different manifestation of the crash.

In normal package builds, it came up as “Shader compiler returned a non-zero error code (-1073741819).”

By setting bAllowCompilingThroughWorkers=False in BaseEngine.ini and modifying ShaderVersion.ush, we were able to catch the crash with a debugger and got this info:

Exception thrown at 0x00007FFECEB5EF86 (dxcompiler.dll) in UnrealEditor-Cmd.exe: 0xC0000005: Access violation reading location 0x00000155DF714D68.

Stack trace:

[Inline Frame] dxcompiler.dll!llvm::Value::getValueID() Line 379 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl<llvm::GlobalVariable,llvm::Value,void>::doit(const llvm::Value & Val) Line 697 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_cl<llvm::GlobalVariable,const llvm::Value>::doit(const llvm::Value & Val) Line 76 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_wrap<llvm::GlobalVariable,const llvm::Value,const llvm::Value>::doit(const llvm::Value & Val) Line 123 C++

[Inline Frame] dxcompiler.dll!llvm::isa(const llvm::Value & Val) Line 134 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl<llvm::GlobalObject,llvm::Value,void>::doit(const llvm::Value & Val) Line 715 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_cl<llvm::GlobalObject,const llvm::Value>::doit(const llvm::Value & Val) Line 76 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_wrap<llvm::GlobalObject,const llvm::Value,const llvm::Value>::doit(const llvm::Value & Val) Line 123 C++

[Inline Frame] dxcompiler.dll!llvm::isa(const llvm::Value & Val) Line 134 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl<llvm::GlobalValue,llvm::Value,void>::doit(const llvm::Value & Val) Line 709 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_cl<llvm::GlobalValue,llvm::Value *>::doit(const llvm::Value * Val) Line 83 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_wrap<llvm::GlobalValue,llvm::Value *,llvm::Value *>::doit(llvm::Value * const & Val) Line 123 C++

[Inline Frame] dxcompiler.dll!llvm::isa_impl_wrap<llvm::GlobalValue,const llvm::Use,llvm::Value *>::doit(const llvm::Use & Val) Line 113 C++

[Inline Frame] dxcompiler.dll!llvm::isa(const llvm::Use & Val) Line 134 C++

[Inline Frame] dxcompiler.dll!llvm::dyn_cast(llvm::Use & Val) Line 294 C++

dxcompiler.dll!`anonymous namespace’::GlobalDCE::GlobalIsNeeded(llvm::GlobalValue * G) Line 244 C++

dxcompiler.dll!`anonymous namespace’::GlobalDCE::runOnModule(llvm::Module & M) Line 100 C++

dxcompiler.dll!`anonymous namespace’::MPPassManager::runOnModule(llvm::Module & M) Line 1667 C++

dxcompiler.dll!llvm::legacy::PassManagerImpl::run(llvm::Module & M) Line 1771 C++

dxcompiler.dll!`anonymous namespace’::EmitAssemblyHelper::EmitAssembly(clang::BackendAction Action, llvm::raw_pwrite_stream * OS) Line 761 C++

dxcompiler.dll!clang::EmitBackendOutput(clang::DiagnosticsEngine & Diags, const clang::CodeGenOptions & CGOpts, const clang::TargetOptions & TOpts, const clang::LangOptions & LOpts, llvm::StringRef TDesc, llvm::Module * M, clang::BackendAction Action, llvm::raw_pwrite_stream * OS) Line 791 C++

dxcompiler.dll!clang::BackendConsumer::HandleTranslationUnit(clang::ASTContext & C) Line 195 C++

dxcompiler.dll!clang::ParseAST(clang::Sema & S, bool PrintStats, bool SkipFunctionBodies) Line 166 C++

dxcompiler.dll!clang::FrontendAction::Execute() Line 468 C++

dxcompiler.dll!DxcCompiler::Compile(const DxcBuffer * pSource, const wchar_t * * pArguments, unsigned int argCount, IDxcIncludeHandler * pIncludeHandler, const _GUID & riid, void * * ppResult) Line 978 C++

UnrealEditor-ShaderFormatD3D.dll!InnerDXCCompileWrapper(TRefCountPtr<IDxcCompiler3> & Compiler, TRefCountPtr<IDxcBlobEncoding> & TextBlob, const wchar_t * * Arguments, unsigned int NumArguments, bool & bOutExceptionError, TRefCountPtr<IDxcResult> & OutCompileResult) Line 504 C++

UnrealEditor-ShaderFormatD3D.dll!DXCCompileWrapper(TRefCountPtr<IDxcCompiler3> & Compiler, TRefCountPtr<IDxcBlobEncoding> & TextBlob, const FDxcArguments & Arguments, TRefCountPtr<IDxcResult> & OutCompileResult) Line 540 C++

UnrealEditor-ShaderFormatD3D.dll!D3DCompileToDxil(const char * SourceText, const FDxcArguments & Arguments, TRefCountPtr<IDxcBlob> & OutDxilBlob, TRefCountPtr<IDxcBlob> & OutReflectionBlob, TRefCountPtr<IDxcBlobEncoding> & OutErrorBlob, TRefCountPtr<IDxcBlob> & OutPdbBlob, FString & OutPdbName, DxcShaderHash & OutHash) Line 636 C++

UnrealEditor-ShaderFormatD3D.dll!CompileAndProcessD3DShaderDXC(const FShaderCompilerInput & Input, const FString & PreprocessedShaderSource, const FString & EntryPointName, const FShaderParameterParser & ShaderParameterParser, const wchar_t * ShaderProfile, ED3DShaderModel ShaderModel, bool bProcessingSecondTime, FShaderCompilerOutput & Output) Line 825 C++

UnrealEditor-ShaderFormatD3D.dll!CompileD3DShader(const FShaderCompilerInput & Input, const FShaderPreprocessOutput & InPreprocessOutput, FShaderCompilerOutput & Output, const FString & WorkingDirectory, ED3DShaderModel ShaderModel) Line 1541 C++

UnrealEditor-RenderCore.dll!FInternalShaderCompilerFunctions::InvokeCompile(const IShaderFormat * Compiler, FShaderCompileJob & Job, const FString & WorkingDirectory, FString & OutExceptionMsg, FString & OutExceptionCallstack) Line 1969 C++

UnrealEditor-RenderCore.dll!FInternalShaderCompilerFunctions::CompileShaderInternal(const IShaderFormat * Compiler, FShaderCompileJob & Job, const FString & WorkingDirectory, FString & OutExceptionMsg, FString & OutExceptionCallstack, int * CompileCount) Line 1996 C++

UnrealEditor-RenderCore.dll!CompileShader(const TArray<IShaderFormat const *,TSizedDefaultAllocator<32>> & ShaderFormats, FShaderCompileJob & Job, const FString & WorkingDirectory, int * CompileCount) Line 2108 C++

UnrealEditor-Engine.dll!FShaderCompileUtilities::ExecuteShaderCompileJob(FShaderCommonCompileJob & Job) Line 34 C++

UnrealEditor-Engine.dll!FShaderCompileThreadRunnable::CompileDirectlyThroughDll() Line 851 C++

UnrealEditor-Engine.dll!FShaderCompileThreadRunnable::CompilingLoop() Line 1332 C++

UnrealEditor-Engine.dll!FShaderCompileThreadRunnableBase::Run() Line 324 C++

[Attachment Removed]

4

I believe there are two different issues reported here. The one from Pablo I believe to be some allocator mismatches that occur when freeing things on shutdown - we have actually reproduced and fixed this locally, and I have a pending CL with new DXC binaries that I hope to submit in the next day or two. I’ll follow up once they are submitted with a CL#.

The crash reported by Matt on the other hand we have not seen before to my knowledge; I am asking around internally to see if it looks familiar to anyone (I know we made some custom modifications/optimizations to DXC in that time frame that could be related). I’ll follow up when I get more info here. We’ve stopped using Incredibuild internally in favour of UBA so if it’s IB-specific it might be hard for us to diagnose, unfortunately.

[Attachment Removed]

5

Yeah, your crash actually happens during the compile step (which means the SEH that Pablo mentioned _should_ be catching it; unclear why it’s not).

I’ve recently been doing multiple full shader rebuilds of a large unnamed internal project (you can probably guess what it is), albeit with UBA for distribution, and haven’t seen this particular crash. Which leads me to believe it’s potentially content-specific.

Does the issue occur with specific shaders, or is it entirely random? Were you able to get a copy of the ShaderDebugInfo files for one of the crashes? I’d be curious if you’d eventually hit it just running SCW directly in a loop with the debug artifacts. If you run with r.DumpShaderDebugInfo=2 (this should be the default), if we’re lucky you might already have these debug files for some broken shaders in your Saved/ShaderDebugInfo folder…

[Attachment Removed]

6

We saw the crash on two separate projects that don’t share materials, and since some full shader rebuilds were fine while identical full (and partial) shader builds failed, we suspected memory corruption or threading issues in the compiler, not any specific shader. I also noticed that there are dxcompiler changes in Unreal’s main branch after 5.7, so it’s possible that the issue was resolved by one of those.

One factor that may be important is that both of our projects that ran into this have Substrate enabled (but not ray tracing), in case that helps narrow it down. I did not find any ShaderDebugInfo associated with the crash (it’s possible the error kills the process before they can be written out). Are there any build configurations that could provide more insight? For us, the most informative step was catching it with the debugger with bAllowCompilingThroughWorkers=False, but that took multiple runs over several days (ran it over the weekend).

[Attachment Removed]

7

The debug info in 5.7 should be dumped by the cooker/editor process, not the compile process itself (at least, the artifacts that I care about here should be). Can you confirm you have r.DumpShaderDebugInfo set to 2? If you could attach logs of a cook or editor process that hit the error that could maybe shed some light too.

[Attachment Removed]

8

I restored the 5.7 version of dxcompiler, cleared ShaderDebugInfo, modified ShaderVersion.ush, and ran a package build, which ran into the issue. r.DumpShaderDebugInfo is at its default value of 2. The cook output around the error was simply:

LogCook: Display: Cooked packages 6013 Packages Remain 17016 Total 23029

LogShaderCompilers: Error: Shader compiler returned a non-zero error code (-1073741819).

LogCook: Display: Cooked packages 6027 Packages Remain 17002 Total 23029

ShaderDebugInfo only produced three files:

PCD3D_SM6\WorldGridMaterial_b9f7ede93ddae9c\Default\DDCKey-Game.txt

VM\NS_Ambient_FloatingLetters\Letters\ParticleSpawnScriptInterpolated\VectorVMPreprocessOutput.txt

VM\NS_Ambient_FloatingLetters\Letters\ParticleUpdateScript\VectorVMPreprocessOutput.txt

[Attachment Removed]

9

I am starting to suspect that you are actually hitting the same crash as Pablo, and the crash you hit when running without workers was unrelated. Anything crashing in the actual compile step should be caught and reported by the normal SCW SEH error reporting path, so the fact that it’s not seems to indicate to me that it’s a different issue. For whatever it’s worth I have long theorized that there’s a stomp somewhere in the cook/editor process, as we’ve been seeing very rare random crashes all over the place during the editor side of shader compilation/preprocessing/etc that we’ve been unable to pinpoint as yet - maybe you hit one of these when running all compile steps in-process.

One further thing I can suggest to try is use corelimit set to a fairly low value (8 or 16) then run the same process that’s crashing as normal (without Incredibuild, just using local workers) and use the VS Child Process Debugging Tool to hopefully catch it in the debugger and see what the callstack looks like there; I’ve had some success with this before, you just need to limit the amount of processes spawned as VS can’t handle a ton at once without choking.

Failing that you can just try the DXC version with the two shutdown crashes fixed; I will be submitting that today and can follow up with a CL when it’s ready.

[Attachment Removed]

10

The fixes and new binaries are now in UE5 Main: 50106448

[Attachment Removed]

11

Ah, sorry to muddy the waters - the error and workaround were identical, so I associated the two. Even though we use Incredibuild most of the time, we have seen the issue come up for local-only builds, and even single threaded bAllowCompilingThroughWorkers=False shader builds. However, it’s intermittent - we can go days without running into it (and it took 4 single threaded full shader rebuilds before we reproed the issue).

[Attachment Removed]

12

Thank you, this has worked so far for 5 full shader rebuilds. I’ll integrate it so our build machines can test it further.

[Attachment Removed]

13

After a number of local tests, I can also confirm the new binaries seemingly address the problem on our end. I’ll make them available to the wider team and keep an eye on stability, will report back if something changes. Thanks!

[Attachment Removed]