Hidden Dependencies, ModifyCook

We are getting a lot of these type of warning spam during cooks.

LogCook: Display: [CookWorker 2]: Unsolicited package /Game/SeventhCurseBP/ProcGen/MapData2/RoomL_1door_A was loaded by package /Game/SeventhCurseBP/ProcGen/MapData2/VLI/RoomL_1door_A_V_2_V. This is a hidden dependency, causes poor cook performance, and it might be a bug to add it to the cook. Declare this package in the AssetRegistry dependencies of the loading package, or mark up its load with FCookLoadScope to specify whether it is runtime or editoronly.

I know these log entries are Display logs, and not warnings, but the message they are conveying certainly comes across more in the spirit of a warning

This spam comes from assets that haven’t changed the way they are cooked in several years now.

  • These assets are part of a procedural level instance based dungeon generation system.
  • These assets(levels) are referenced by data tables, by TSoftObjectPtr<UWorld>
  • These data tables, are included by a ModifyCook overriden asset manager

So while the references to this world are hidden in the sense that those assets are indirectly referenced by the cooked data tables, I don’t understand why this warrants these messages that suggests something bad/unexpected is occurring. I was under the impression that the inclusion of asset reference chains is a normal function of the cooking process. I am explicitly including a data table package from the ModifyCook function, therefore, any downstream references from that link in the chain, are in fact solicited.

Judging by the part

Declare this package in the AssetRegistry dependencies of the loading package, or mark up its load with FCookLoadScope to specify whether it is runtime or editoronly.

  • Declare this package…
    • How? And why isn’t the fact that the data table references all these levels with soft references already providing that?
  • or mark up its load with
    • ModifyCook doesn’t actually load packages, it just returns a list of them, so FCookLoadScope doesn’t seem appropriate here either

Here is how my ModifyCook function works

void UGameAssetManager::ModifyCook(TConstArrayView<const ITargetPlatform*> TargetPlatforms, TArray<FName>& PackagesToCook, TArray<FName>& PackagesToNeverCook)
{
    Super::ModifyCook(TargetPlatforms, PackagesToCook, PackagesToNeverCook);

    IAssetRegistry& LocalAssetRegistry = GetAssetRegistry();

    const TArray<FName> TagsToCook {
       FName(TEXT("ForceCook")),
       FName(TEXT("LobbyMap"))
    };

    TMap<FString, int32> ForceCookCountsByAssetPaths;
    
    for (auto Tag : TagsToCook)
    {
       // make sure we cook all potential procedural prop data
       FARFilter Filter;
       Filter.bRecursiveClasses = false;
       Filter.bIncludeOnlyOnDiskAssets = true;
       Filter.TagsAndValues.Add(Tag);

       TArray<FAssetData> AssetDatas;
       if (LocalAssetRegistry.GetAssets(Filter, AssetDatas))
       {
          for (const FAssetData& Asset : AssetDatas)
          {
             PackagesToCook.Add(Asset.PackageName);

             FString PreviewPath;
             if(Asset.GetTagValue(TEXT("LobbyMapPreview"), PreviewPath))
             {
                PackagesToCook.Add(FName(PreviewPath));
                
                ForceCookCountsByAssetPaths.FindOrAdd(TEXT("Opaque"))++;
             }
          }
       }

       UE_LOG(LogGameAssetManager, Display, TEXT("UGameAssetManager::ModifyCook Added %d assets with tag %s"), AssetDatas.Num(), *Tag.ToString());       
    }
}

I conditionally apply the “ForceCook” asset tag in OnGetExtraObjectTagsWithContext delegate functions.

Did something change with the ModifyCook process to make it play nice with more recent changes to the engine and avoid this spam?

Adding the packages in your ModifyCook function is unrelated to the issue the log statement is reporting. You are correct in your understanding of ModifyCook: the cooker adds all of the packages you list there to its list of packages to cook (load and save), and additionally it does a graph search of the runtime dependencies in the AssetRegistry starting from the packages you add, and adds all the packages found in the graph search to the list of packages to cook. If your SoftObjectPaths are UsedInGame dependencies, they will be discovered in that graph search and will be added to the cooker’s list of packages to cook; this occurs before any packages are loaded.

The reported problem occurs when the cooker reaches the package in its list of packages to cook, and loads and saves it. Unreal allows arbitrary c++ code to execute for the class instances loaded by a package; during a cook some of this c++ code is activated, by the cooker calling Serialize, PostLoad, and PreSave. Those c++ functions might load packages. The log message is reporting that the one of the packages so loaded was unexpected because it was not declared as the right kind of dependency in the AssetRegistry dependencies.

In 5.5 and earlier we used a different calculation for whether the load was expected; that calculation was whether any package traversed during the graph search reported the package being loaded, and if so then we marked the load as expected and did not report it. But that definition of expected was too broad, and caused us to miss declaration of some hidden dependencies, so we changed it to suppress the message only if the the load was expected specifically from the dependencies reported by the package that loaded the target package. Hidden dependencies as noted in the message are a problem for incremental cooks, which need to know about all packages that contain data that changes the bytes of the cooked package that loads them.

You said your dependencies are stored as SoftObjectPaths in your source package, and that does add them to the graph search, but it does NOT add them to the expected list of packages loaded by the cooked package. SoftObjectPaths are not loaded automatically by the linker, they are only loaded when class-specific code calls FSoftObjectPath.TryLoad, or some other manual call to a load function of the SoftObjectPath. Therefore by default we expect them not to load when the source package loads, and give the warning unless the class loading them declares them as expected.

Your class might need them to load. Maybe you need to read data from them as part of the cook-Save of your package. But in that case, you should mark them as an expected load. If you expect them to be used in game as well, you should mark them up as FCookLoadScope (ECookLoadType::UsedInGame), otherwise mark them up as FCookLoadScope (ECookLoadType::EditorOnly).

The load of the packages must be coming from c++ code in one of your classes in the source package. You can find that callstack by instrumenting code in FPackageTracker::OnCreatePackage, to hit a breakpoint when PackageName == FName(TEXT(“/Game/SeventhCurseBP/ProcGen/MapData2/RoomL_1door_A”)), and then run the cook commandlet in a mode that cooks only the single source package:

UnrealEditor.exe <ProjectName> -run=cook -targetplatform=<PlatformName> -cooksinglepackagenorefs -package=/Game/SeventhCurseBP/ProcGen/MapData2/VLI/RoomL_1door_A_V_2_V

Is that callstack to OnCreatePackage the stack that is loading the package /Game/SeventhCurseBP/ProcGen/MapData2/RoomL_1door_A, or is it loading some other package?

You can see LoadPackage calls in the middle of the stack; that is similar to what I was expecting to see, but I was expecting it to come from a class that you had written, rather than WorldPartition. It looks like the WorldPartition call to LoadPackage is loading an Actor package, so I expect the name of the package being loaded is something inside of an __ExternalActors__ directory rather than your normal asset RoomL_1door_A. External actor packages loaded by WorldPartition are marked as expected loads by a naming convention in ProcessUnsolicitedPackage, that is why they do not emit the warning.

FAsyncLoadingThread2, and the other variants of it that are used at runtime or in different editor configurations, is not supposed to load SoftObjectPaths when it is asked to load a package. It is only supposed to load imports, aka hard dependencies, aka TObjectPtr (or variants of TObjectPtr) that were serialized when saving the package. And all of those imports are supposed to be parsed by the AssetRegistry and listed as hard references (either game or editoronly) so the ContainsDependency call made in ProcessUnsolicitedPackages is supposed to mark them as expected. SoftObjectPaths are only supposed to be loaded from system-specific code (outside of the FAsyncLoadingThread2).

Maybe this load is unrelated, and it is coming from a different package that does have a hard import of the package. A load from a package with a hard import does not cause the display message to fire, but because of where we put the breakpoint we might be looking at that call which is unrelated to the call that is causing the problem.

To test that hypothesis: look in the local variables in OnCreatePackage when OnCreatePackage is called on RoomL_1door_A.

That function calls FInstigator Instigator = GetPackageCreationInstigator();.

That instigator has a field Referencer that contains the name of the package that the cooker thinks caused the load of the Actor. What is the Instigator in the case of the load of RoomL_1door_A? Is it the problematic referencer, RoomL_1door_A_V_2_V?

If the load is coming from RoomL_1door_A_V_2_V, then this is a bug in either FAsyncLoadingThread2, or in the PackageAccessTracking code that is supposed to tell the cooker which package is doing the importing of the package being loaded.

If the load is coming from something else, then we will need to change your cooksinglepackage repro case and the location of the breakpoint to find out what is causing the load from package RoomL_1door_A_V_2_V, rather than telling us about this other package that loads RoomL_1door_A.

The problem is not that the packages are unsolicited. The word unsolicited is a historic term that is now a misnomer. The problem is that the packages are loaded when we don’t expect them to be loaded: we expect them to be loaded when the cooker decides it is time to cook them, not when some other package loads that is unrelated to them.

In this case it seems like its the cooker’s expectations that have a bug, rather than the systems in the package having a bug and failing to declare that the loads are expected.

This looks like a bug in the cooker’s handling of instanced packages. The AssetRegistry does not have any data for an instanced package; when an instanced package loads dependencies and the cooker wants to know whether the dependencies were expected, it needs to look up the name of the package that was loaded from rather than the name of the package in memory. There is code in the cooker to map the instanced package names to the loaded path so it can do that lookup. It looks like in this case,

The actor package /Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5 was loaded instanced into the package /Temp/Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V_b9121be533f5a01a_9d33dd8756627c9f_InstanceOf_/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5.

This instanced load was a subtask of the top level instanced load of the map /Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V as an instanced sublevel into the a temporary package for the map that uses an instance of it (/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/VLI/RoomL_1door_A_V_2_V).

When loading this actor package, it has a hard import (not a soft reference as you thought; that soft reference is elsewhere, and the actor has a hard import) of the target package /Game/SeventhCurseBP/ProcGen/MapData2/RoomL_1door_A.

The cooker is supposed to map /Temp/Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V_b9121be533f5a01a_9d33dd8756627c9f_InstanceOf_/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5 back to /Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5, see the hard import in the loaded-from actor package’s AssetRegistry dependencies, and understand that the load is expected.

Instead, the cooker is mapping /Temp/Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V_b9121be533f5a01a_9d33dd8756627c9f_InstanceOf_/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5 to something we don’t know yet, and whatever that is does not declare a dependency on the targetpackage. The cooker then unhelpfully tells us that the package that is doing the loading without declaring the reference is the top level map package.

So (1) there’s a definite usefulness problem in the diagnostic information and (2) there is a very probable bug in the calculation of whether the reference is unexpected.

To confirm that the referencer does declare these correctly, can you open the referencing actor in the referencer viewer and look at its hard dependencies? You need to open the base map, find the actor in the Outliner, right-click it and select “Open Actor in Referencer Viewer”, and in the referencer viewer, select the view filters that shows only the hard references (see screenshot of me doing that for an Actor in Lyra).

I’m just working on 5.6.1 and we are also seeing this, is it possible that maybe assets need to be re-saved? Quite often with engine upgrades these go away after re-saving them but I haven’t tried that yet

I do feel like there is some bug in this code:

In our case I see a lot of these:

/Game/WwiseAudio/Events/Amb/p_amb_trainYard_emt_alarm_far_02 was loaded by package /Game/Maps/MAP_NAME/MAP_NAME. This is a hidden dependency, causes poor cook performance, and it might be a bug to add it to the cook. Declare this package in the AssetRegistry dependencies of the loading package, or mark up its load with FCookLoadScope to specify whether it is runtime or editoronly.

and if I do a reference viewer its just the map referencing the asset and the wwise event does not reference anything (we have about 12,000 of these in our logs through multiprocess cook of 4)

I reproduced the problem locally (and added test content for it in EngineTest), and have prepared a fix. It’s around 100 lines in the two PackgaeTracker files.

> Cooker: SkipOnlyEditorOnly: Fix incorrect reporting of hidden dependencies when an instanced package is loaded with imports of its own. We report all of those transitive dependencies as dependencies of the package loading the instanced package, but we were not correctly merging the flag that states whether the transitive import was an expected load with the flag that states whether the instanced package was an expected load.

Do you want a diff to try or would you rather wait for me to submit and you can cherrypick the change from github?

Fix submitted in CL 44929591, aka github commit 94bd60a6db86d5f7a96582ade5393cc6b6e4f6db.

> Cooker: SkipOnlyEditorOnly: Fix incorrect reporting of hidden dependencies when an instanced package is loaded with imports of its own. We report all of those transitive dependencies as dependencies of the package loading the instanced package, but we were not correctly merging the flag that states whether the transitive import was an expected load with the flag that states whether the instanced package was an expected load.

Let me know if that doesn’t fix the issue, or if other spam is still occurring.

You can also turn off the spam; I turned it off with a code change after 5.6 shipped (changed the log severity from `Display` to `Verbose`), because there are too many instances to fix for now and it fires many time from the game code of some of our test projects. I hope to have time to fix those before 5.7 ships and reenable it to Display level.

What are the callstacks in OnCreatePackage of one or some or all of the remaining ones? If they’re coming from engine classes call LoadPackage, or if they’re still coming from FAsyncLoadingThread2, then I want to fix them.

IKRetarger: Fixed by CL 44939486 aka github commit https://github.com/EpicGames/UnrealEngine/commit/cab7792da74cdac4d23fdebb9b8e476977fd0f57\.

Automatic Serialize upgrades of old UPROPERTY TSoftObjectPtr to a UObject* do a LoadPackage, need to mark the LoadPackage as expected.

Water: Fixed by CL 45414753 aka github commit https://github.com/EpicGames/UnrealEngine/commit/b3551a40ab2c64cdc6af452d1630fde5d81e3337

WaterBodyComponent Serialize changes nullptr into a default object, need to mark the default object as a UsedInGame load.

Clouds: The callstack shows the LoadPackage comes from the AsyncLoader, when WorldPartition is loading an external actor for a map, which based on our experience with the previous error indicates a bug in the instancing code. The log message indicates it was loaded by /Game/SeventhCurse/Maps/PostProcessing/LI_BaseOverworldLight. What kind of asset is LI_BaseOverworldLight?

Mesh: The callstack shows the LoadPackage call comes from the cooker’s load of an independent asset, but this does not match the log message, which indicates it was loaded from a generated package; a generated package does not call LoadPackage during that callstack. Is it possible the callstack was captured from a different asset that also loaded hp_d_1x2a_SM, rather than being captured during the save of /Game/SeventhCurseBP/Maps/VS1/VS1_House3?

DaySequenceActor: thanks, I should have thought to search for m_SimpleVolumetricCloud_Inst in code.

It is loaded by the constructor for ACelestialVaultDaySequenceActor::ACelestialVaultDaySequenceActor.

We will need to move that out of the constructor into an initialization hook that only gets called if the loaded VolumetricCloudComponent does not override the property. I’ll get a fix.

I also need to investigate why the callstack didn’t mention the ACelestialVaultDaySequenceActor constructor.

Thanks for your speedy response.

Where would I use FCookLoadScope (ECookLoadType::UsedInGame)? ModifyCook hasn’t traditionally loaded assets, it just expects PackagesToCook.Add(…)

My ModifyCook function just returns the name of the packages I find via a simple registry query, and adds those package names to the out parameter.

The data table references the level like so in each record.

// In the data table record
UPROPERTY(BlueprintReadOnly, VisibleAnywhere)
TSoftObjectPtr<UWorld> Level;
  • Do I need to actually load the data table asset in ModifyCook , manually iterate the records, and Level.LoadSynchronous() within the scope of a FCookLoadScope (ECookLoadType::UsedInGame)?
  • Or would resolving the data table alone, inside the scope of a UsedInGame, cascade to the soft references contained within the data table rows? I would hope so, as that is the only thing that really makes sense here. A user can’t possibly account for every nested soft asset that goes from(in this case) data table -> levels -> literally hundreds of other actors.
  • Is there a meta tag that can be applied to my TSoftObjectPtr<UWorld> Level; that can identify that specific property resolution as used in game or editor?
  • Also I forgot to mention that the entire MapData2 folder is also cooked by asset registry rule, so it seems the modifycook dependency from the data table is really a secondary references to these levels.
  • [Image Removed]
  • So it’s even more puzzling why the maps within that folder generate these warnings
  • [Image Removed]
  • Static meshes, etc referenced by maps in the cook list are the furthest from unsolicited packages you can get.
  • I suppose the indirect reference via the data table isn’t part of the issue here then?

Here’s the callstack for a single package build, though I’m not sure what I’m looking for with this.

UE::cook::FPackageTracker::OnCreatePackage(FName) PackageTracker.cpp:208

UE::cook::FPackageTracker::NotifyUObjectCreated(const UObjectBase *, int) PackageTracker.cpp:163

FUObjectArray::AllocateUObjectIndex(UObjectBase *, EInternalObjectFlags, int, int, FRemoteObjectId) UObjectArray.cpp:320

UObjectBase::AddObject(FName, EInternalObjectFlags, int, int, FRemoteObjectId) UObjectBase.cpp:258

UObjectBase::UObjectBase(UClass *, EObjectFlags, EInternalObjectFlags, UObject *, FName, int, int, FRemoteObjectId) UObjectBase.cpp:159

StaticAllocateObject(const UClass *, UObject *, FName, EObjectFlags, EInternalObjectFlags, bool, bool *, UPackage *, int, FRemoteObjectId, FGCReconstructionGuard *) UObjectGlobals.cpp:3854

StaticConstructObject_Internal(const FStaticConstructObjectParameters &) UObjectGlobals.cpp:4960

[Inlined] NewObject(UObject *, FName, EObjectFlags, UObject *, bool, FObjectInstancingGraph *, UPackage *) UObjectGlobals.h:1921

FAsyncPackage2::CreateUPackage() AsyncLoading2.cpp:10690

FAsyncPackage2::InitializeLinkerLoadState(const FLinkerInstancingContext *) AsyncLoading2.cpp:6459

FAsyncPackage2::ImportPackagesRecursiveInner(FAsyncLoadingThreadState2 &, FIoBatch &, FPackageStore &, FAsyncPackageHeaderData &) AsyncLoading2.cpp:6399

FAsyncPackage2::ImportPackagesRecursive(FAsyncLoadingThreadState2 &, FIoBatch &, FPackageStore &) AsyncLoading2.cpp:6432

FAsyncLoadingThread2::FinishInitializeAsyncPackage(FAsyncLoadingThreadState2 &, FAsyncPackage2 *) AsyncLoading2.cpp:4860

FAsyncPackage2::ProcessLinkerLoadPackageSummary(FAsyncLoadingThreadState2 &) AsyncLoading2.cpp:6806

FAsyncPackage2::Event_ProcessPackageSummary(FAsyncLoadingThreadState2 &, FAsyncPackage2 *, int) AsyncLoading2.cpp:7317

FEventLoadNode2::Execute(FAsyncLoadingThreadState2 &) AsyncLoading2.cpp:5713

FAsyncLoadEventQueue2::ExecuteSyncLoadEvents(FAsyncLoadingThreadState2 &) AsyncLoading2.cpp:5919

FAsyncLoadingThread2::ProcessAsyncLoadingFromGameThread(FAsyncLoadingThreadState2 &, bool &) AsyncLoading2.cpp:9215

[Inlined] FAsyncLoadingThread2::TickAsyncThreadFromGameThread(FAsyncLoadingThreadState2 &, bool &) AsyncLoading2.cpp:9946

FAsyncLoadingThread2::TickAsyncLoadingFromGameThread(FAsyncLoadingThreadState2 &, bool, bool, double, TArrayView<…>, bool &) AsyncLoading2.cpp:9565

FAsyncLoadingThread2::FlushLoading(TArrayView<…>) AsyncLoading2.cpp:11244

FlushAsyncLoading(TArrayView<…>) AsyncPackageLoader.cpp:362

FlushAsyncLoading(int) AsyncPackageLoader.cpp:331

LoadPackageInternal(UPackage *, const FPackagePath &, unsigned int, FLinkerLoad *, FArchive *, const FLinkerInstancingContext *, const FPackagePath *) UObjectGlobals.cpp:1779

LoadPackage(UPackage *, const FPackagePath &, unsigned int, FArchive *, const FLinkerInstancingContext *, const FPackagePath *) UObjectGlobals.cpp:2143

LoadPackage(UPackage *, const wchar_t *, unsigned int, FArchive *, const FLinkerInstancingContext *) UObjectGlobals.cpp:2119

FWorldPartitionLevelHelper::LoadActorsInternal(FWorldPartitionLevelHelper::FLoadActorsParams &&, FWorldPartitionLevelHelper::FLoadedPropertyOverrides &&) WorldPartitionLevelHelper.cpp:927

FWorldPartitionLevelHelper::LoadActorsWithPropertyOverridesInternal(FWorldPartitionLevelHelper::FLoadActorsParams &&) WorldPartitionLevelHelper.cpp:514

[Inlined] FWorldPartitionLevelHelper::LoadActors(FWorldPartitionLevelHelper::FLoadActorsParams &&) WorldPartitionLevelHelper.cpp:621

UWorldPartitionRuntimeLevelStreamingCell::OnPrepareGeneratorPackageForCook(TArray<…> &) WorldPartitionRuntimeLevelStreamingCell.cpp:439

UWorldPartition::PrepareGeneratorPackageForCook(IWorldPartitionCookPackageContext &, TArray<…> &) WorldPartitionCook.cpp:94

FWorldPartitionCookPackageSplitter::PopulateGeneratorPackage(ICookPackageSplitter::FPopulateContext &) WorldPartitionCookPackageSplitter.cpp:157

UE::cook::FGenerationHelper::TryCallPopulateGeneratorPackage(TArray<…> &) CookGenerationHelper.cpp:1189

UCookOnTheFlyServer::BeginCacheObjectsToMove(UE::cook::FGenerationHelper &, UE::cook::FCookGenerationInfo &, UE::cook::FCookerTimer &, TArray<…> &) CookOnTheFlyServer.cpp:3750

UCookOnTheFlyServer::PrepareSaveGenerationPackage(UE::cook::FGenerationHelper &, UE::cook::FPackageData &, UE::cook::FCookerTimer &, bool) CookOnTheFlyServer.cpp:3637

UCookOnTheFlyServer::PrepareSaveInternal(UE::cook::FPackageData &, UE::cook::FCookerTimer &, bool, UE::cook::ESuppressCookReason &) CookOnTheFlyServer.cpp:4239

UCookOnTheFlyServer::PrepareSave(UE::cook::FPackageData &, UE::cook::FCookerTimer &, bool, UE::cook::ESuppressCookReason &) CookOnTheFlyServer.cpp:4039

UCookOnTheFlyServer::PumpRuntimeSaves(UE::cook::FTickStackData &, unsigned int, int &, bool &) CookOnTheFlyServer.cpp:4951

[Inlined] IsEngineExitRequested() CoreGlobals.h:404

UCookOnTheFlyServer::TickMainCookLoop(UE::cook::FTickStackData &) CookOnTheFlyServer.cpp:1528

UCookOnTheFlyServer::TickCookByTheBook(const float, ECookTickFlags) CookOnTheFlyServer.cpp:1406

UCookCommandlet::RunCookByTheBookCook(UCookOnTheFlyServer *, void *, ECookByTheBookOptions) CookCommandlet.cpp:596

UCookCommandlet::CookByTheBook(const TArray<…> &) CookCommandlet.cpp:548

UCookCommandlet::Main(const FString &) CookCommandlet.cpp:266

FEngineLoop::PreInitPostStartupScreen(const wchar_t *) LaunchEngineLoop.cpp:3860

[Inlined] FEngineLoop::PreInit(const wchar_t *) LaunchEngineLoop.cpp:4159

[Inlined] EnginePreInit(const wchar_t *) Launch.cpp:40

GuardedMain(const wchar_t *) Launch.cpp:143

GuardedMainWrapper(const wchar_t *) LaunchWindows.cpp:128

LaunchWindowsStartup(HINSTANCE__ *, HINSTANCE__ *, char *, int, const wchar_t *) LaunchWindows.cpp:282

WinMain(HINSTANCE__ *, HINSTANCE__ *, char *, int) LaunchWindows.cpp:339

Yes, that is specifically a breakpoint for that named single cook package if (PackageName == FName(TEXT(“/Game/SeventhCurseBP/ProcGen/MapData2/RoomL_1door_A”)))

The LoadPackage higher in the callstack is the loading of a level instance actor referenced by ALevelInstance

For example, the First LoadPackage, called from Load FWorldPartitionLevelHelper::LoadActorsInternal

__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5/Game/../../../Game/Content/ntLevel.StaticMeshActor_7So for clarity

  • /Game/SeventhCurseBP/ProcGen/MapData2/VLI/ is included in the cook map paths
  • SeventhCurseBP/ProcGen/MapData2/Base/ is not included in the cook map paths. (In fact, many levels we reference via LevelInstanceActor are not)
  • /Game/SeventhCurseBP/ProcGen/MapData2/VLI/ may reference level instance actors that do not exist in the cook map paths
  • Those level instances may reference actors/packages that do not exist in the forced cook paths.

Given that an ALevelInstance will often reference worlds that will be outside the special package paths, how does the system maintain that an expected ALevelInstance actor also has expected dependencies on the TSoftObjectPtr<UWorld> references they hold?

On my limited understanding, it looks like the package expectation might be getting lost across ALevelInstance world references.

Instigator = {UE::Cook::FInstigator} {Referencer="/Temp/Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V_b9121be533f5a01a_9d33dd8756627c9f_InstanceOf_/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5", Category=Unsolicited}
 Referencer = {FName} "/Temp/Game/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V_b9121be533f5a01a_9d33dd8756627c9f_InstanceOf_/Game/__ExternalActors__/SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V/2/IA/5RB8UH382NXG04LE5AQ9D5"
 Category = {UE::Cook::EInstigator} Unsolicited

The referencer is the level instance actor that references SeventhCurseBP/ProcGen/MapData2/Base/RoomL_1door_A_Base_V MapData2/Base level that is only ever referenced by one the variant maps that are in the cook path

[Image Removed]

Essentially

  • Levels in SeventhCurseBP\ProcGen\MapData2\VLI (in the cook path)
  • Have ALevelInstance actors that reference maps in SeventhCurseBP\ProcGen\MapData2\Base (not in the include cook path)

Surely the TSoftObjectPtr<UWorld> references in a ALevelInstance should be treated as a solicited package. Or there should be something that forces it as such.

There’s also the added complexity that ALevelInstance, depending on ELevelInstanceRuntimeBehavior, may or may not embed that map into the containing map. In this case, the base map is embedded in the variant map at cook time, but many of our other ALevelInstance, we use in standalone mode, which means that the TSoftObjectPtr<UWorld> will point to levels that aren’t inside the special paths.

I realize I could probably silence many of these warnings with adding the SeventhCurseBP\ProcGen\MapData2\Base folder to the map cook path but that would just be covering up what seems like a clear bug. And it still wouldn’t account for many of the other warnings of the same type that are about maps referenced by ALevelInstance from arbitrary path locations. User folders, prototype folders, etc. Folders we don’t want to blanketly include in our cooks map folder list.

In our client build log, we are getting over 900 of these “…This is a hidden dependency…” warnings, and it looks like all of them are associated with nested level instances

Thanks for the explanation. I see now that ALevelInstance implements some functions that gather those soft reference into dependencies (ALevelInstance::OnCookEvent, GetSoftReferencedContentObjects)

Here is the reference view to the actor. Hopefully this tells us something useful. It’s still too fuzzy in my head to have an expectation of whether I’m seeing what I should be seeing here.

[Image Removed]

I’m fine cherry picking from a github changelist.

Thanks again for your time in working this out.

Edit: Unless you want me to verify the change results in our situation before you commit it or something.

Should be fine to cherry pick from github, the logs being spammed is quite a lot