Intermittent cook failures when a PCGWorldActor tries to register ALandscapeProxy components in non-world partition mode.

Development Programming & Scripting
, ,
1

We’ve seen intermittent cook errors in 5.6.1. Looks like it’s some kind of timing/GC fight between stricter behaviour checking in AActorComponent and the actual behaviour of PCGWorldActor with landscape proxies…

  • The actual error which causes the cook to fail is the check “if (!IsValid(InWorld) || InWorld->IsCleanedUp())” in UActorComponent::RegisterComponentWithWorld. “IsCleanedUp” is returning true, which logs an error and returns, forcing the actual check(bAllRegistered) to subsequently fail.
  • This is being called from APCGWorldActor::BeginCacheForCookedPlatformData, which calls ProxyToRegister->RegisterAllComponents() for each ALandscapeProxy in the world IF both of the following are true…
    • LandscapeCacheObject->SerializationMode == EPCGLandscapeCacheSerializationMode::SerializeOnlyAtCook
    • World->GetWorldPartition() returns null.

In the case of our cook, the code path through APCGWorldActor::BeginCacheForCookedPlatformData into ProxyToRegister->RegisterAllComponents() happens 100% of the time the map is cooked, but only results in the underlying error approx. 25-50% of the time. This has been observed on several different PCs, both in manual packaging from editor and automated CI build processes.

I’ve combed bug reports, UE 5.7 release notes etc, and browsed latest engine source, and haven’t found anything that looks like a “smoking gun” or closely adjacent issue.

As an interim workaround, we’ve demoted the error to a warning ONLY during the cook, on the assumption that the behaviour was previously the same for us in UE 5.5.4 and it’s just the tightening of the ActorComponent check that has surfaced it in 5.6.1.

[Attachment Removed]

2

Steps to Reproduce
Repro is uncertain and likely content specific:

Cook a map containing a PCGWorldActor and a landscape when not using world partition. Cook MAY fail with critical error and callstack below. In our case the error happens only on one of many maps, but whether this map is unique in PCGWorldActor / landscape / world partition is unknown.

LogWindows: Error: appError called: Assertion failed: bAllRegistered [File:D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Runtime\Engine\Private\Actor.cpp] [Line: 6005]

10:06:53

10:06:53

10:06:53 LogWindows: Error: begin: stack for UAT

10:06:53 LogWindows: Error: === Critical error: ===

10:06:53 LogWindows: Error:

10:06:53 LogWindows: Error: Assertion failed: bAllRegistered [File:D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Runtime\Engine\Private\Actor.cpp] [Line: 6005]

10:06:53 LogWindows: Error:

10:06:53 LogWindows: Error:

10:06:53 LogWindows: Error:

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8b9c2dcf8 UnrealEditor-Core.dll!FDebug::CheckVerifyFailedImpl2() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Runtime\Core\Private\Misc\AssertionMacros.cpp:728]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8b4f13241 UnrealEditor-Engine.dll!AActor::RegisterAllComponents() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Runtime\Engine\Private\Actor.cpp:6005]

10:06:53 LogWindows: Error: [Callstack] 0x00000173784b1913 UnrealEditor-PCG.dll!APCGWorldActor::BeginCacheForCookedPlatformData() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Plugins\PCG\Source\PCG\Private\PCGWorldActor.cpp:155]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb40dff UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::RouteBeginCacheForCookedPlatformData() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:12614]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb06061 UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::CallBeginCacheOnObjects() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:4377]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb35fb1 UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::PrepareSaveInternal() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:4131]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb357a2 UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::PrepareSave() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:4039]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb3a564 UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::PumpRuntimeSaves() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:5031]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb4953d UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::TickMainCookLoop() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:1528]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8adb48c5b UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::TickCookByTheBook() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\CookOnTheFlyServer.cpp:1408]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8ac7d0e3d UnrealEditor-UnrealEd.dll!UCookCommandlet::RunCookByTheBookCook() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\Commandlets\CookCommandlet.cpp:598]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8ac791d0a UnrealEditor-UnrealEd.dll!UCookCommandlet::CookByTheBook() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\Commandlets\CookCommandlet.cpp:550]

10:06:53 LogWindows: Error: [Callstack] 0x00007ff8ac7ae6c3 UnrealEditor-UnrealEd.dll!UCookCommandlet::Main() [D:\BuildAgent\work\e036aca36fe06e9c\Engine\Source\Editor\UnrealEd\Private\Commandlets\CookCommandlet.cpp:269]

… (stack truncated for character limit) …

10:06:53 LogWindows: Error:

10:06:53 LogWindows: Error: end: stack for UAT

[Attachment Removed]

3

Hey Andrew,

this is not something we’ve seen before and the fact that it only happens on a single map is at least a bit suspicious.

At this points we’re unsure in which scenario the world would ever be being cleaned up at the same time while the cooker is running regular saves.

We’ll likely need a bit more info to get to the bottom of this.

First, for the crash, is there a chance you can catch that check in the debugger and provide us with the call stacks for all threads?

Visual studio has a “parallel stacks window” where you can save a picture of all the stacks. Alternatively you can export them to a file from the regular debugger window.

A full log from such a run could also useful, to make sure there are no other messages that could be relevant to the problem.

As for the specific map and your PCG usage:

If you don’t need the landscape cache at runtime you can disable the serialization and that should help with reducing cooked data and should avoid the error. Landscape cache serialization defaulted to enabled before 5.6 but is now disabled by default. So there’s a chance if this map is older that is has it enabled while your other maps might not have it.

Are you using runtime generation together with any of the advanced landscape data from the cache, e.g. reading landscape layer textures?

If you don’t need that and only sample points you might alternatively use the world ray hit node to generate points on your landscape, which only uses the physics data.

Best,

Sebastian

[Attachment Removed]