UE5 BuildCookRun consumes all available memory and crashes

Development Platform & Builds
, , , , ,
1

I’m working on upgrading an existing UE4 production to UE5 (5.3.2). Everything has gone well so far as getting the engine and game up and running for our daily users, but I’m now trying to get our build machines updated too.

This is where I’m running into memory issues on our build agent. A few minutes into BuildCookRun, the UnrealEditor process has consumed a significant amount of memory, along with several ShaderCompileWorkers, and eventually runs out and crashes. The machine is a cloud-hosted dedicated VPS with 32 GB of RAM.

For context, running BuildCookRun on the UE4 build tops out around 24 GB. Here’s the (anonymized) command I’m running via a Powershell build step on the build machine;

./RunUAT.bat BuildCookRun `
    -nop4 -buildmachine `
    -project="<FullPathToGame>/Game.uproject" `
    -platform=Win64 `
    -clientconfig=Shipping `
    -build -cook `
    -archive -archivedirectory=Archive `
    -stage -pak -prereq -compressed

(similar to the one described in this talk)

I’ve tried adjusting MemoryMinFreePhysical / MemoryMinFreeVirtual in DefaultEditor.ini under CookSettings to reserve a larger buffer of free memory.

I set it as high as 16 GB at one point to test and while it did seem to garbage collect more often in the logs, it still eventually crashed on OOM.

I also tried to limit MemoryMaxUsedPhysical and MemoryMaxUsedVirtual to all the way down 16 GB. This had practically identical results.

Finally, I’ve also tested all of this on my local machine, which has 64 GB of RAM. I am able to complete the cook on this machine, but it uses almost all of the available memory to do so, coming withing a few GB of running out of memory.

Has anyone run into similar issues? Any luck working around it?

1 Like