5.8 ACL Database crash

Development Character & Animation
, ,
1

We are seeing this crash on 5.8.1 when the editor closes, FACLDatabaseCompressedAnimData::~FACLDatabaseCompressedAnimData() attempts to mark the codec database asset dirty. But the codec has already been marked for destroy and is invalid, so hits the below check. Unfortunately this then leads to a crash as the object index is invalid. This happens when the codec is destroyed before an AnimationSequence that uses it (which I believe can happen normally given non-determinism of GC order in Unreal).

To repro:

  • Create a new ACL database
  • Create new bone compression settings using that ACL database
  • Associate it with any AnimationSequence asset
  • Close the editor
    [Attachment Removed]
2

Steps to Reproduce
To repro:

  • Create a new ACL database
  • Create new bone compression settings using that ACL database
  • Associate it with any AnimationSequence asset
  • Close the editor

[Attachment Removed]

3

Callstack:

[Inline Frame] UnrealEditor-CoreUObject.dll!FUObjectArray::IndexToObject(int) Line 1083	C++
 	UnrealEditor-CoreUObject.dll!FUObjectArray::AllocateSerialNumber(int Index) Line 531	C++
 	UnrealEditor-CoreUObject.dll!FWeakObjectPtr::operator=(FObjectPtr ObjectPtr) Line 51	C++
 	[Inline Frame] UnrealEditor-ACLPlugin.dll!FWeakObjectPtr::{ctor}(FObjectPtr) Line 85	C++
 	[Inline Frame] UnrealEditor-ACLPlugin.dll!FWeakObjectPtr::{ctor}(const UObject *) Line 88	C++
 	[Inline Frame] UnrealEditor-ACLPlugin.dll!TWeakObjectPtr<UAnimationCompressionLibraryDatabase,FWeakObjectPtr>::{ctor}(UAnimationCompressionLibraryDatabase *) Line 63	C++
 	UnrealEditor-ACLPlugin.dll!EditorDatabaseMonitor::MarkDirty(UAnimationCompressionLibraryDatabase * Database) Line 72	C++
 	[External Code]	
 	UnrealEditor-Engine.dll!TSparseArray<TSparseSetElement<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>::DestroyElements(int Index, int Count) Line 997	C++
 	[Inline Frame] UnrealEditor-Engine.dll!TSparseArray<TSparseSetElement<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>::DestroyElements() Line 986	C++
 	[Inline Frame] UnrealEditor-Engine.dll!TSparseArray<TSparseSetElement<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>::Empty(int) Line 671	C++
 	UnrealEditor-Engine.dll!TSparseArray<TSparseSetElement<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>::~TSparseArray<TSparseSetElement<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>>,TSparseArrayAllocator<TSizedDefaultAllocator<32>,FDefaultBitArrayAllocator>>() Line 538	C++
 	[Inline Frame] UnrealEditor-Engine.dll!TSet<TTuple<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>>,TDefaultMapHashableKeyFuncs<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>,0>,FDefaultSetAllocator>::{dtor}() Line 111	C++
 	[Inline Frame] UnrealEditor-Engine.dll!TMapBase<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>,FDefaultSetAllocator,TDefaultMapHashableKeyFuncs<FIoHash,TUniquePtr<FCompressedAnimSequence,TDefaultDelete<FCompressedAnimSequence>>,0>>::{dtor}() Line 70	C++
 	UnrealEditor-Engine.dll!UAnimSequence::~UAnimSequence() Line 1930	C++
 	[External Code]	
 	UnrealEditor-CoreUObject.dll!FObjectPurge::DestroyObjects(bool bUseTimeLimit, double TimeLimit, double StartTime) Line 1008	C++
 	UnrealEditor-CoreUObject.dll!IncrementalDestroyGarbage(bool bUseTimeLimit, double TimeLimit) Line 5273	C++
 	UnrealEditor-CoreUObject.dll!IncrementalPurgeGarbage(bool bUseTimeLimit, double TimeLimit) Line 4908	C++
 	UnrealEditor-CoreUObject.dll!PurgeAllUObjectsOnExit() Line 6641	C++
 	UnrealEditor-CoreUObject.dll!StaticExit() Line 6004	C++
 	[Inline Frame] UnrealEditor-CoreUObject.dll!Invoke(void(*)() &) Line 46	C++
 	[Inline Frame] UnrealEditor-CoreUObject.dll!UE::Core::Private::Tuple::TTupleBase<TIntegerSequence<unsigned int>>::ApplyAfter(void(*)() &) Line 264	C++
 	UnrealEditor-CoreUObject.dll!TBaseStaticDelegateInstance<void __cdecl(void),FDefaultDelegateUserPolicy>::ExecuteIfSafe() Line 723	C++
 	[Inline Frame] UnrealEditor.exe!TMulticastDelegateBase<FDefaultDelegateUserPolicy>::Broadcast() Line 306	C++
 	UnrealEditor.exe!TMulticastDelegate<void __cdecl(void),FDefaultDelegateUserPolicy>::Broadcast() Line 1135	C++
 	UnrealEditor.exe!FEngineLoop::AppPreExit() Line 7318	C++
 	UnrealEditor.exe!FEngineLoop::Exit() Line 5294	C++
 	[Inline Frame] UnrealEditor.exe!EngineExit() Line 71	C++
 	[Inline Frame] UnrealEditor.exe!GuardedMain::__l2::EngineLoopCleanupGuard::{dtor}() Line 132	C++
 	UnrealEditor.exe!GuardedMain(const wchar_t * CmdLine) Line 211	C++
 	UnrealEditor.exe!LaunchWindowsStartup(HINSTANCE__ * hInInstance, HINSTANCE__ * hPrevInstance, char * __formal, int nCmdShow, const wchar_t * CmdLine) Line 287	C++
 	UnrealEditor.exe!WinMain(HINSTANCE__ * hInInstance, HINSTANCE__ * hPrevInstance, char * pCmdLine, int nCmdShow) Line 359	C++

[Attachment Removed]

4

Hi, thanks for reporting this. It looks like we should change Codec to be a TWeakObjectPtr<const UAnimBoneCompressionCodec_ACLDatabase> in order to prevent the crash that you’re running into. Then we should catch if the data is marked for GC in the destructor like this:

#if WITH_EDITORONLY_DATA
FACLDatabaseCompressedAnimData::~FACLDatabaseCompressedAnimData()
{
	// Our compressed data is being destroyed which means either we are unloading or we are about to have
	// new compressed data. If the new codec isn't an ACL instance we have no way of knowing so assume the
	// database is dirty and double check.
	if (const UAnimBoneCompressionCodec_ACLDatabase* CodecPtr = Codec.Get())
	{
		EditorDatabaseMonitor::MarkDirty(CodecPtr->DatabaseAsset);
	}
}
#endif

Are you able to test that change and let me know if it resolves the issue? You’ll need to fix up some other code in that file after changing the type, so I have a shelf at 57890831 for reference.

On a related note, did you run into any issues when you started using the ACL database? We don’t use it internally and previously a blocking bug came up which hasn’t been resolved yet - it’s listed here.

[Attachment Removed]

5

We tried out your change and that worked for us, thank you!

We haven’t yet tried it with multiprocess cook, but we do use that on some machines, so we will likely run into that next.

[Attachment Removed]

6

Great, thanks for confirming the change resolved the crash. I’ll get that submitted into the 5.9 release.

In terms of the cooking bug, just be aware that could be a potential blocker if it is still an issue (it was reported a while back) and you run into it. Let me know if you want to discuss that further, otherwise it sounds like we’re good to close out this thread?

[Attachment Removed]

7

From some local testing it is currently working for us with multiprocess cook. We’re all good to close this out, thanks!

[Attachment Removed]

8

Great, that’s good news!

[Attachment Removed]