When our project crashes in a packaged build, everything is dumped to the files correctly with symbols. However, in editor builds, the symbol information is missing from the crash dump information. We have DetailedCallstacksInNonMonolithicBuilds set to true and everything suggests we should be seeing the symbols but can’t. Any help would be appreciated
Steps to Reproduce
Run editor outwith visual studio, type “debug crash” and wait for crash dump to be made. Open up CrashContext.runtime-xml in the crashes folder and scroll down to the “PCallStack” section. See that the callstack doesn’t have symbols. E.g.
<PCallStack>
UnrealEditor-Engine 0x00007ffad5570000 + 26bad07
UnrealEditor-Engine 0x00007ffad5570000 + 2681993
UnrealEditor-Engine 0x00007ffad5570000 + 266a86b
UnrealEditor-Core 0x00007ffadac00000 + 48173c
UnrealEditor-Engine 0x00007ffad5570000 + 2669834
UnrealEditor-UnrealEd 0x00007ffad1fa0000 + 1539d62
UnrealEditor-UnrealEd 0x00007ffad1fa0000 + 1458143
UnrealEditor-UnrealEd 0x00007ffad1fa0000 + 1457f0f
UnrealEditor-OutputLog 0x0000020e390c0000 + 96229
UnrealEditor-OutputLog 0x0000020e390c0000 + 6f071
UnrealEditor-Slate 0x00007ffad4f00000 + 23f191
UnrealEditor-Slate 0x00007ffad4f00000 + 299866
UnrealEditor-Slate 0x00007ffad4f00000 + 280ca1
UnrealEditor-Slate 0x00007ffad4f00000 + 3d8195
UnrealEditor-Slate 0x00007ffad4f00000 + 3be435
UnrealEditor-Slate 0x00007ffad4f00000 + 3c05a2
UnrealEditor-Slate 0x00007ffad4f00000 + 3ccf12
Hi,
For the Windows Editor, the callstack in CrashContext.runtime-xml is set by CrashReportClientEditor.exe a.k.a CRC (resolved in bool FCrashDebugHelperWindows::CreateMinidumpDiagnosticReport( const FString& InCrashDumpFilename )). By default, the Editor launches one of the CRC binary found at start up, it prefers the Shipping one (Engine\Binaries\Win64\CrashReportClientEditor.exe) but it would use the Development or Debug one as fallback if one is available. CRC will run in ‘monitor’ mode. When the Editor crash, the Editor process will copy the logs and write information in CrashContext.runtime-xml and then signal CRC on a pipe to remote stack walk the crashing process. When CRC is done, it signals the Editor to continue. The Editor is free to die, it will optionally log the crash call stack (see UE_LOG_CRASH_CALLSTACK) and terminates, but it doesn’t update CrashContext.runtime-xml after the fact. Also note that the Editor it doesn’t update the logs that were copied in the Crash folder either, so the log in the crash folder doesn’t contain the resolved call stack.
On its side, CRC will get signaled on Pipe to process a crash, it will then remove stack walk the Editor process, gather the callstack, resolve the callstack and update CrashContext.runtime-xml and it will send the files found in the crash folder (Crashes\UECC-Windows-FD75AAE44E4F2DD84363D8BAE9CBE4E2_0000).
It was implemented this way to increase the chance to collect the crashes because resolving a callstack in a crashing process has some chances crash the process again… because this needs memory and CPU.
If the Editor cannot launch CRC because the executable doesn’t exist (or it has a problem launching), then you will not get the resolved callstack. If you have CrashReportClientEditor.exe compiled and available, you can start the Editor, then attach the debugger to CrashReportClientEditor.exe that should run in background and you can debug it to figure out what’s wrong. If you don’t any CRC binary, then you probably need to compile it (Win64/Shipping), that would probably solve your issue.
Regards,
Patrick
Thanks for the detailed reply. When I built CrashReporClientEditor.exe that has fixed the issue. Now I just need to figure out how to add that to my BuildGraph to get UGS to distribute it out to the team
Nice! If you need help adding CRC to your build graph let us know, but it should be relatively simple. This file has an example if you need one: //UE5/Release-5.5/Engine/Build/Graph/Tasks/BuildAndTestProject.xml
Regards,
Patrick
Great, thanks. I found the entry for it and it verified that my attempt at adding it in was very close. All working now 