[BUG][COOK SOFT LOCK] Cook can be soft locked due to the bug in the logic related to the UAnimSequence::bUseRawDataOnly

Development Platform & Builds
, ,
1

Hello,

After switching to the UE 5.5 we started to experience occasional cook soft locks: at some point of execution the cooker stop making any progress and just endlessly reports “Cooked packages Nc Packages Remain Nr Total Nt” where Nc, Nr and Nt never change.

We were able to fully investigate the issue and we have found that there is a bug in the logic related to the UAnimSequence::bUseRawDataOnly that causes these soft locks. Please find some details below. Also, see the provided repro project.

[Simplified] description of the issue:

1. If there are two animation sequences: VictimAnim and CulpritAnim, and

2. VictimAnim->RefPoseSeq (UAnimSequence::RefPoseSeq) points to the CulpritAnim, and

3. VictimAnim is loaded at the time T1 during the cooking, and

4. All async “compilation” tasks (FAnimationSequenceAsyncCacheTask; see the UAnimSequence::BeginCacheDerivedData(const ITargetPlatform* TargetPlatform)) are finished for the VictimAnim at the time T2 during the cooking, and

5. UAnimSequence::FlagDependentAnimationsAsRawDataOnly() is called for the CulpritAnim at the moment T3 during the cooking, and

6. T3 > T2, and VictimAnim is not yet saved and not yet scheduled for saving by the cook logic at the time T3, then

7. Cook will be soft locked at the attempt to save the VictimAnim later, because VictimAnim->IsCachedCookedPlatformDataLoaded() will never return true

The reason the VictimAnim->IsCachedCookedPlatformDataLoaded() will never return true after the moment T3 is the following:

1. At the moment T2 (see p.4 above) the VictimAnim will be switched to the following state:

VictimAnim->bUseRawDataOny == false && VictimAnim->DataKeyHash == VictimAnim->CreateDerivedDataKeyHash(<TargetPlatform>) && VictimAnim->CacheTasksByKeyHash.Contains(VictimAnim->CreateDerivedDataKeyHash(<TargetPlatform>)) == false && VictimAnim->CompressedData.IsValid(VictimAnim) == trueThis is an expected normal state of the object and VictimAnim->IsCachedCookedPlatformDataLoaded() will return true when VictimAnim in this state.

2. But at the moment T3 (see p.5 above) the value of VictimAnim->bUseRawDataOnly will be changed to true, as a result of execution of UAnimSequence::FlagDependentAnimationsAsRawDataOnly() for the CulpritAnim. So the state of the VictimAnim will change to the following:

VictimAnim->bUseRawDataOny == true && <== JUST CHANGED VictimAnim->DataKeyHash == VictimAnim->CreateDerivedDataKeyHash(<TargetPlatform>) && VictimAnim->CacheTasksByKeyHash.Contains(VictimAnim->CreateDerivedDataKeyHash(<TargetPlatform>)) == false && VictimAnim->CompressedData.IsValid(VictimAnim) == trueThis state of an animation sequence object can’t be properly handled by the IsCachedCookedPlatformDataLoaded() and IsCachedCookedPlatformDataLoaded() always return false for the object in this state (and this state is never changed as a result of IsCachedCookedPlatformDataLoaded() execution).

Please see the next post (post characters limit) …

2

Steps to Reproduce
1. Unzip the attached project (infckbug.zip)

2. Generate Visual Studio project files for the repro project (infckbug/infckbug.uproject)

3. Open the generated project solution (infckbug/infckbug.sln)

4. Build the project

5. Open the infckbug/cook-with-bug-conditions-enforced-V2.bat for editing and update the path to the UnrealEditor-Cmd.exe to match the engine setup on your PC (the provided repro project was developed for the launcher version of the UE 5.5.4)

6. Update the path to the UnrealEditor-Cmd.exe (see step 5 above) also in the following files:

* infckbug/cook-with-bug-conditions-enforced-V2-wait-for-debugger.bat

* infckbug/cook-with-bug-conditions-enforced-V3-in-code-enforcement.bat

* infckbug/cook-with-bug-conditions-enforced-V3-in-code-enforcement-wait-for-debugger.bat

7. Run the infckbug/cook-with-bug-conditions-enforced-V2.bat

8. Observe that cook is soft locked and will never finish (you need to wait 2 mins to see the message marked below):

[Image Removed]

DISCLAIMER A: Due to the nature of the issue, this step may not repro the issue on every PC. On some PC this cook may finish normally. We did our best to prepare this repro (infckbug/cook-with-bug-conditions-enforced-V2.bat) to trigger the issue on most PCs, but it still may not trigger on every PC.

If this cook finished normally (no lock) on your PC, please try on some other PC(s) (better with different hardware)

DISCLAMER B: Due to the same nature of the issue, it is extremely hard to repro with regular project cook. This repro cook is run by the specially crafted command line. This command line “simulates” the conditions that trigger the issue during the regular cook logic execution.

9. If you was not able to repro with step 7 above even on different PCs, please run the infckbug/cook-with-bug-conditions-enforced-V3-in-code-enforcement.bat

10. Observe that cook is soft locked and will never finish (you need to wait 2 mins to see the message marked below):

[Image Removed]

DISCLAMER C: This repro method is a special approach to the repro. It does not try to simulate the issue conditions during the cook logic execution (as it may not work reliably on every PC, see disclaimers A & B above). Instead, this repro approach directly executes in code the sequence of actions that causes the issue, before the cook logic execution. This sequence of actions is exactly the same sequence that causes the issue during the regular cook execution.

Please see the infckbug/Source/infckbug/TestEngineSubsystem.cpp => UTestEngineSubsystem::OnCommandletPreMain()

11. To catch the issue in action:

* Set the breakpoint in the infckbug/Source/infckbug/TestEngineSubsystem.cpp => UTestEngineSubsystem::OnCommandletPreMain() to step through the sequence of actions that causes the issue.

*Run the infckbug/cook-with-bug-conditions-enforced-V3-in-code-enforcement-wait-for-debugger.bat

* Attach the debugger to the executed UnrealEditor-Cmd.exe process

* Wait for the triggering of the breakpoint you set in the UTestEngineSubsystem::OnCommandletPreMain() (see above)

* See the comments in the UTestEngineSubsystem::OnCommandletPreMain() to get the information about the issue triggering mechanism

3

… continuation of the original topic post (post characters limit)

Possible crash outcome case:

Instead of the cook soft locking, this bug may also manifest itself as a termination of the cook process due to the assertion failure in the UAnimSequence::PreSave(FObjectPreSaveContext ObjectSaveContext):

`Error: Assertion failed: IsCachedCookedPlatformDataLoaded(ObjectSaveContext.GetTargetPlatform()) [File:<…>\Engine\Source\Runtime\Engine\Private\Animation\AnimSequence.cpp] [Line: 975]

[Callstack] UnrealEditor-Engine.dll!UAnimSequence::PreSave()
[Callstack] UnrealEditor-CoreUObject.dll!UE::SavePackageUtilities::CallPreSave()
[Callstack] UnrealEditor-CoreUObject.dll!anonymous namespace'::RoutePresave() [Callstack] UnrealEditor-CoreUObject.dll!UPackage::Save2() [Callstack] UnrealEditor-CoreUObject.dll!UPackage::Save() [Callstack] UnrealEditor-UnrealEd.dll!UEditorEngine::Save() [Callstack] UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::SaveCookedPackage() [Callstack] UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::PumpSaves() [Callstack] UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::TickMainCookLoop() [Callstack] UnrealEditor-UnrealEd.dll!UCookOnTheFlyServer::TickCookByTheBook() [Callstack] UnrealEditor-UnrealEd.dll!UCookCommandlet::RunCookByTheBookCook() [Callstack] UnrealEditor-UnrealEd.dll!UCookCommandlet::CookByTheBook() [Callstack] UnrealEditor-UnrealEd.dll!UCookCommandlet::Main()This happens when VictimAnim is scheduled for the saving (by the cook logic) between the times T2 and T3 (see the problem description in the original topic post). We faced this outcome during the working on the provided repro, but this case is almost impossible to repro reliably on diffrent PCs, so we can’t provide a repro for this outcome (it is also impossible to simulate this outcome in code, as we did for the original soft lock outcome).

Since this issue has a negative impact on our team’s productivity, we would like to know:

1. If this issue is known, and if it is known, is it planned to be fixed ?

2. What can be the best way for us to fix or workaround the issue at the moment ?

Thanks in advance for any feedback on this!

5

Hi Alexey!

Thanks for the detailed break-down and repro project - that helped to narrow it down. So the issue is that once FlagDependentAnimationsAsRawDataOnly is called is only set the bRawDataOnly flag and doesn’t clear out the compressed data itself. (which is conditional in UAnimSequence::BeginCacheDerivedData so it just early-outs rather than recompressing the data.

Either change:

PRAGMA_DISABLE_DEPRECATION_WARNINGS Seq->bUseRawDataOnly = true; PRAGMA_ENABLE_DEPRECATION_WARNINGSto

Seq->ClearAllCachedCookedPlatformData();

or adjust the condition in UAnimSequence::BeginCacheDerivedData

if (KeyHash.IsZero() || (DataKeyHash == KeyHash && (CompressedData.IsValid(this) || CacheTasksByKeyHash.Contains(KeyHash))) || DataByPlatformKeyHash.Contains(KeyHash))to

if (KeyHash.IsZero() || (DataKeyHash == KeyHash && PRAGMA_DISABLE_DEPRECATION_WARNINGS !bUseRawDataOnly && PRAGMA_ENABLE_DEPRECATION_WARNINGS (CompressedData.IsValid(this) || CacheTasksByKeyHash.Contains(KeyHash))) || DataByPlatformKeyHash.Contains(KeyHash))It is also worth noting that most of this has been refactored in 5.6 (bUseRawDataOnly is not longer used, and more threadsafety was added).

Hope this helps out,

Cheers,

Jurre

6

Hey Jurre!

Thank you for the provided possible solutions! Much appreciated! We were thinking about the first approach, but we were not sure if this will be the correct refactoring. Thank you again!

Best regards, Alexey

PS May I ask you a favor to take a look at the another issue I’ve posted some time ago ? Or maybe you can ask one of your colleagues to take a look at this issue ? The link to this another issue: [Content removed] ?

Thanks in advance!