We’re currently trying to implement cumulative patching in our project to reduce the size of files developers have to download to get new builds. We’re able to successfully do this by using commands such as:
BuildCookRun -Project=Path\To\My.uproject -Cook -NoP4 -Stage -UTF8Output -Package -Prereqs -CrashReporter -Pak -BuildMachine -NoCodeSign -WaitForUATMutex -Build -CreateReleaseVersion=1 -Platform=Win64 -ClientConfig=Development -Client // Do a week's worth of development work BuildCookRun -Project=Path\To\My.uproject -Cook -NoP4 -Stage -UTF8Output -Package -Prereqs -CrashReporter -Pak -BuildMachine -NoCodeSign -WaitForUATMutex -Build -CreateReleaseVersion=2 -BasedOnReleaseVersion=1 -AddPatchLevel -Platform=Win64 -ClientConfig=Development -ClientThis gives us an extra set of pak/ucas/utoc files, of the form My-WindowsClient_0_P.ucas, so this is functioning as expected.
However, as part of doing this, we get one error logged for every asset that was present when release 1 was created that has been removed when release 2 is being created, of the form:
LogCook: Warning: Could not resolve package /Game/MyAssetINoLongerRequire from ../../../My/Releases/1/WindowsClient/AssetRegistry.binIt’s a normal part of ongoing development to remove assets, especially for games that are not yet released, so this behaviour is surprising to me. The cause of this seems to be https://github.com/EpicGames/UnrealEngine/blob/ab87f26b09009438a4cf01cd765187c6b93b142d/Engine/Source/Editor/UnrealEd/Private/CookOnTheFlyServer.cpp#L11680 , where the parameter bVerifyPackagesExist is being set to true - it seems like this should be set to false to me, to allow for content deleted as part of ongoing development. (Edit: Setting this value to false causes warnings about failing to cook the removed files, so this is not sufficient to fix the issue.)
Is there something I’m missing here? Are we using the wrong arguments here to try and get cumulative patching working? It would be great to get some documentation on using -AddPatchLevel on https://dev.epicgames.com/documentation/en-us/unreal-engine/how-to-create-a-patch-in-unreal-engine?application_version=5.6#howtocreateapatchusingthecommandline as this currently only discusses the scenario where a single release is created and used to create patches.
That code dates back to 2014; I’m doing some research to find out why it was required. Possibly it is no longer required. If you comment out the entire block and never call GetAllPackageFilenamesFromAssetRegistry at all, does your patched game work correctly, without cook errors?
`// Remove this if statement and body:
if (!IsCookingDLC() && !BasedOnReleaseVersion.IsEmpty())
{
// if we are based on a release and we are not cooking dlc then we should always be creating a new one (note that we could be creating the same one we are based on).
// note that we might erroneously enter here if we are generating a patch instead and we accidentally passed in BasedOnReleaseVersion to the cooker instead of to unrealpak
UE_CLOG(CreateReleaseVersion.IsEmpty(), LogCook, Fatal, TEXT(“-BasedOnReleaseVersion must be used together with either -dlcname or -CreateReleaseVersion.”));
// if we are creating a new Release then we need cook all the packages which are in the previous release (as well as the new ones)
for (const ITargetPlatform* TargetPlatform : TargetPlatforms)
{
// if we are based of a cook and we are creating a new one we need to make sure that at least all the old packages are cooked as well as the new ones
FString OriginalAssetRegistryPath = GetBasedOnReleaseVersionAssetRegistryPath(BasedOnReleaseVersion, TargetPlatform->PlatformName()) / GetAssetRegistryFilename();
…
}
}`
Your patch commandline arguments look correct to me; here are my notes about patchlevel:
-addpatchlevel: An important field for cumulative patching; Finds the highest existing _P_<Integer> label in the release directory, and sets the pak files created for the current release’s differences to use <Integer>+1. See use of ShouldAddPatchLevel in CopyBuildToStagingDirectory.Automation.cs
Modifying the public documentation requires a more polished version of the notes, with more research and summary of what the code is doing; no eta for when we can get that added to the docs.
Fortunately the author of the code was around to tell me what I was missing.
The behavior of requiring all previous packages is put there to handle references that might exist from DLC. A project might have a package in the base game, then ship some DLCs based on that base game, and the DLCs might have references to the package as well. The project might then remove the references to the package from the other base game packages and push out a patch. We do not consider (and currently we do not have any tools that allow considering) references to packages that come from DLCs. If we removed the package from the patched base game, the DLCs would then start failing on the patch due to the missing package.
This is not a problem for your case because you do not have any published DLCs, but before we change the behavior for the engine we need to develop a solution that will, in a bulletproof manner, let a project know that they have to keep a package because it is referenced from a DLC.
Removing the block you suggested, creating a release with an added patch level, and then building a client using that release all seems to work fine, yeah - the game cooks fine and also seems to run fine. I also tried this after removing the AssetRegistry.bin and Metadata/DevelopmentAssetRegistry.bin from the Releases directory, so they no longer seem necessary at all.
Thanks very much!