This is less of an issue and more of a general question. Are there currently any known reasons or risks where we might want to avoid enabling multi-process cook on all of our CI/CD jobs? We’ve enabled it on our CI jobs but have held off so far on enabling it for nightly QA builds or builds that we’ll be shipping due to concerns over data integrity. After a bit of internal thought, we think we might be being overly cautious at the detriment of our cook performance. I know for experimental features such as incremental cook we should avoid it for published builds, but I’m curious if MP cook is considered safe to use on published builds (or really all builds).
One concern we had was over pak file shuffling if assets weren’t built in a deterministic order (which could hurt patch sizes), but that’s only a concept. Is there any reason to actually be concerned about this?
Our other concern is more of a hardware concern, which is whether we can expand the memory available on all machines to support what’s needed for MP cook, but we’re still evaluating options there.
MP cook is considered safe at this point and the cooked data should be identical to a single process cook. Regarding performance, it’s actually a YMMV situation. One main factor is that WP levels currently have to be cooked on the main process and the cells cannot be distributed. Games based on large WP level(s) can see less improvement than games based on classic levels. Another important factor, as you already alluded to, is the amount of RAM in the system. You want to avoid too many garbage collection passes which can result in the same assets being constantly loaded and destroyed.
Regarding Pak shuffling, this is unrelated to the cooking step as the pak\iostore is generated later in the packaging process.
In 5.6 we had some issues with a VERY large and complex WorldPartitioned map, unfortunately very hard to reproduce in a smaller more isolated map.
Every time we rebuilt HLODs/Navmesh/MiniMap builds would start failing until we added -NoAssetRegistryCacheDiscovery to the commandline or destroyed the AssetRegistryDiscoveryCache.bin.