We started using incremental cook recently. It did not improve our average cook times; zen import + cook takes pretty much the same time as a full cook did before. So we’re looking at what we can tweak to make it perform better.
Questions:
How often do you export cook snapshots? We currently do it for our postsubmit builds so roughly every 1-2 hours.
Do you run full cooks when you export or is it import - cook - export?
How do you debug incremental cook performance? Any tools we can use?
What effect does building the editor have on the snapshots? Should one preferably not rebuild the editor when prepping new export snapshots?
Hello, before checking in with the team to confirm our internal cadences and recommendations, there are a few details you can check that would help.
Firstly, by default, only the Engine files are allow-listed for incremental cooks. Could you please confirm whether you’ve expanded this allow-list in DefaultEditor.ini to include your project root?
After that check, it would be good to review the cooking logs to see what’s reported about the incremental cook.
You can use the argument “-IncrementalValidate” to check if skipped packages would have produced identical output.
Along with this, you can use a new cooker argument in 5.7 that will report on why packages were recooked, “-CookIncrementallyModifiedDiagnostics”.
It creates the file in “Saved\Cooked\<Platform>\<Project>\Metadata\ModifiedCookedPackages.txt” which lists which dependency was found to be modified for each package.
Could you please check these logs for the “INCREMENTAL COOK” log messages?
I found these similar forum articles that would be a good read, too, as they contain relevant responses from one of our principal engine programmers.
The second article includes a recommendation to regularly run with “-IncrementalValidate” to catch packages that were skipped when they should have been recooked.
On prepping new export snapshots, you shouldn’t need to avoid editor rebuilds.
With iterative cooking, this could invalidate previously cooked packages, but with incremental cooking, the check is against the class schema hash and CustomVersion changes instead.
If you’re still running into issues after checking those settings, could you please share:
the log files from the cooks with the arguments mentioned above
whether you’re using World Partition
if these concerns are for the build farm, or local iterations
Thanks for the excellent response, we’ve basically been busy following your suggestions. Step 1 was to add Allow,<ProjectRoot>. Step 2 was constructing the wiring for cooked AngelScript classes (we use https://github.com/Hazelight/UnrealEngine\-Angelscript); the idea is we cook those if <ProjectRoot> is enabled for incremental cook. If we do all that our cooks seem to be coming down to 2 minutes basically.