Caveat: There may be a lot of work required to get incremental cook working well in 5.6. We have made a lot of improvements during the development of 5.7. If you don’t have the time to fix the problems with it, you may want to wait until 5.7 comes out, and in the mean time use MultiProcessCook as an accelerator of full recooks.
IncrementalCook is not allowed to incrementally skip packages containing types from your project unless you enable it in config. We turn it off by default because project types might contain hidden dependencies on other packages, files, managers containing raw pointers to packages, or other types of inputs that we don’t automatically detect. If any such undiscovered hidden dependencies exist, then updating those inputs would cause a false incremental skip and create an invalid build containing stale data. Most notably, if a c++ class has a raw object pointer (UObject*) that points to a UObject in another package, and it uses data from that UObject* to influence the bytes written in the bytes of its own package, then that will be a hidden dependency. To allow auto-collection of those dependencies, you should convert all of your types to use TObjectPtr. See the section on TObjectPtr in the UE5 Migration Guid: https://dev.epicgames.com/documentation/en\-us/unreal\-engine/unreal\-engine\-5\-migration\-guide. We are detecting and fixing the remaining hidden dependencies in engine types (we have fixed 95% of them in 5.6, and 99% of them in 5.7), but with project types that we don’t have the ability to test we have to turn them off to remain conservative. We don’t yet have a plan for how to improve this, but we know we need to develop one.
Once you are ready to try out the incremental cook of packages containing your types, you can enable it with a config setting. An example of doing this is in the Lyra project:
Samples\Games\Lyra\Config\DefaultEditor.ini
[CookSettings]
+IncrementalClassScriptPackageAllowList=Allow,<ProjectRoot>
See also the notes in Engine\Config\BaseEditor.ini, for the keys IncrementalClassScriptPackageAllowList,IncrementalClassAllowList,IncrementalClassDenyList under CookSettings, for more detailed enable settings.
Part of our ability to allow Engine and internal project types to cook incrementally is to validate that there are no false incremental scripts, using the incremental validate and incremental compare jobs you mentioned. Those jobs are designed to detect false incremental skips, they do not provide a lot of information on false incremental recooks (which is what you were asking about). When run, IncrementalValidate does an incremental cook in the current workspace (you must preserve this workspace between runs for the test to be able to run), and uses the existing -diffonly feature of the cook (used to detect non-determinism) to cook anyway all of the skipped packages, and detect and report differences as errors if they have different results than the previous cook. IncrementalCompare reports less granular information - it only reports which packages are different, not how they are different - but it reports an important bit of information that IncrementalValidate does not - it reports packages that are falsely added or falsely removed from the incremental cook.
To verify the correctness of your incrementalcook, you should run IncrementalValidate regularly after enabling your project’s types using IncrementalClassScriptPackageAllowList.
The number of packages recooked is reported at the end of the cook, in this log line:
LogCook: Display: Packages Cooked: 2, Packages Incrementally Skipped: 3860 Packages Skipped by Platform: 219, Total Packages: 4081
If you enable IncrementalClassScriptPackageAllowList for your types and are still seeing many false incremental recooks after when running two cooks back to back, let me know and I can help you debug it. We have some diagnostic tools coming in 5.7 (preview release coming soon) which will help track them down. The reasons for false recooks in 5.6 are usually indeterminism in the calculations of dependencies.
