Thanks, I believe I see the problem. I’m surprised we never encountered this internally, that’s the only reason I doubt my analysis. Can you try out the fix? I am still testing it locally; it might cause knockon errors. But if you’re getting frequent reproductions of the problem then I expect you will hit those errors quickly and can let me know. Otherwise, if no information from you or my testing about any errors, I will submit the fix into UE5/Main in a few days, for inclusion in 5.6.2 if we make one. If your problem still occurs after the attempted fix, I will submit anyway, but will add some more verbose information and will ask you for a new log to analyze.
What went wrong:
The log reports that the package with the problem started to save on the Director, was retracted from Director and sent to a CookWorker, the CookWorker finished it, and then the Director noticed the CachedObjectsInOuterInfo was unexpectedly non-empty. That sequence of events means that the CookDirector sent the package to SaveStalledAssignedToWorker, possibly after populating CachedObjectsInOuterInfo, and then received the FCookWorkerServer::RecordResults function with the save results from the Cooker. RcordResults does not trigger the cleanup function for the package: FCookGenerationHelper::ResetSaveState. ResetSaveState is the only function that clears CachedObjectsInOuterInfo.
The fix:
RecordResults should call UCookOnTheFlyServer::ReleaseCookedPlatformData on the package before setting its ParentGenerationHelper to nullptr, so that ReleaseCookedPlatformData can call ParentGenerationHelper->ResetSaveState.
Insert this code between BUGFIXSTART and BUGFIXEND into the location in RecordResults where we call SetParentGenerationHelper(nullptr, StateChangeReason):
Engine\Source\Editor\UnrealEd\Private\Cooker\CookWorkerServer.cpp
void FCookWorkerServer::RecordResults(FPackageResultsMessage& Message)
{
...
// If generated or generator, and this is a save or other end-of-cook signal, defer events and mark saved
if (PackageData->IsGenerated() && bTerminalStateChange)
{
TRefCountPtr<FGenerationHelper> ParentGenerationHelper = PackageData->GetOrFindParentGenerationHelper();
if (!ParentGenerationHelper)
{
...
}
else
{
DeferGenerationHelperEvents.Emplace(ParentGenerationHelper);
for (int32 PlatformIndex = 0; PlatformIndex < NumPlatforms; ++PlatformIndex)
{
ITargetPlatform* TargetPlatform = OrderedSessionPlatforms[PlatformIndex];
FPackageRemoteResult::FPlatformResult& PlatformResult = Result.GetPlatforms()[PlatformIndex];
if (!bResultIsSaveResult || PlatformResult.WasCommitted())
{
ParentGenerationHelper->MarkPackageSavedRemotely(COTFS, *PackageData, TargetPlatform, GetWorkerId());
}
}
// BUGFIXSTART
// ReleaseCookedPlatformData would ordinarily be called when we leave the SaveState during
// PromoteToSaveComplete or SendToState below, but we need to call it now before we
// SetParentGenerationHelper(nullptr), so that it can call ParentGenerationHelper->ResetSaveState.
// It is safe to call twice, so call it now and it will be a noop when called again later.
// Call it with EStateChangeReason::DoneForNow and EPackageState::AssignedToWorker so that it does
// the minimum shutdown for leaving the SaveState rather than also doing extra work for packages
// returning to idle (such as SetParentGeneration(nullptr)) that we need to do ourselves in a specific
// order.
COTFS.ReleaseCookedPlatformData(*PackageData, EStateChangeReason::DoneForNow,
EPackageState::AssignedToWorker);
// BUGFIXEND
PackageData->SetParentGenerationHelper(nullptr, StateChangeReason);
}
}
...