Memory leak on editor close after building HLODs

Development Platform & Builds
, ,
1

After updating to 5.7.1, our horde job to build HLODs is unable to complete due to a memory leak when exiting.

The build step itself finishes without any errors, but the following check is getting triggered:

~TLockFreeFixedSizeAllocator()
{
    AutoRTFM::PopAllOnAbortHandlers(this);
    UE_AUTORTFM_ONCOMMIT(this)
    {
       check(GetNumUsed() == 0);
       Trim();
    };
}

I was able to repro the error consistently on horde agents, as well as by running the step locally.

Would you have any pointers as to how we could debug this issue?

Job logs:

LogInit: Display:
LogInit: Display: Success - 0 error(s), 23908 warning(s)
LogInit: Display:
Execution of commandlet took:  2h 39m 26s (9566.79 seconds)
LogAnalytics: Display: [HordeAnalytics.Dev] AnalyticsET::EndSession
LogPython: Display: Object '/ControlRig/Python/RigHierarchy/add_controls_for_selected_PY.ControlOutputFormat' was externally referenced when shutting down Python. Forcibly releasing its Python resources!
LogShaderCompilers: Display: Shaders left to compile 0
LogPackageName: Display: FPackageName: Mount point removed: '../../../XXX/Intermediate/DataprepTemp/' unmounted from '/DataprepCore/Transient/'
Took 9,587.69s to run UnrealEditor-Cmd.exe, ExitCode=-1073741819
Editor terminated with exit code -1073741819 while running WorldPartitionBuilder for D:\Horde\Working\XXX\Sync\XXX\XXX.uproject; see log D:\Horde\Working\XXX\Sync\Engine\Programs\AutomationTool\Saved\Logs\WorldPartitionBuilder-2025.11.29-14.53.51.txt
AutomationTool executed for 2h 39m 49s
AutomationTool exiting with ExitCode=1 (Error_Unknown)
2

Hello!

Can you share the crash report? It should be under Project\Saved\Crashes. There might be multiple reports so you might have to review the logs to know which one comes from HLOD builder command.

You might also want to try CL#47223132. This fix addresses a double free when running commandlets.

Regards,

Martin

3

Did the CL I suggested help? You mentioned that you can reproduce locally, can you attach to the process and capture the stack when the assert fires?

4

I’m not sure why the change hasn’t been considered. My guess is that it’s because changes to Public headers are usually forbidden in hotfixes to avoid breaking the binary compatibility with plugins. This rule is related to the Launcher builds and the Fab plugins. In this case, the change is on private methods so there might be a way to get this fix approved. I’ll look at having this fix part of version 5.7.2.

Regards,

Martin

5

Hi! Unfortunately, it doesn’t seem like any crash report is being generated for that issue.

6

Hi Martin, I’ve ran a batch of tests using the CL you mentionned and so far no issues.

Thank you!

Are there any drawbacks to it? Wondering why it wasn’t integrated in 5.7.