Rare crashes in FExportArchive::SerializeBulkMeta

Development Platform & Builds
, ,
1

Hello,

We have 3 different callstacks related to a crash in

FExportArchive::SerializeBulkMeta(UE::BulkData::Private::FBulkMetaData&, FBulkDataCookedIndex&, long long&, int) (AsyncLoading2.cpp:2779).

From the callstack, the crash can originate from Nanite::FResources::Serialize() (from a StaticMesh / SkeletalMesh) or FDistanceFieldVolumeData::Serialize (from a StaticMesh).

I do not have the names of the related assets unfortunately.

They have been reported by Sentry on a Shipping build, and they have not yet been reproduced by our QA.

I did not find any related thread on EPS or a potential fix on Github, hence I’m asking for some support here, as I don’t have any idea how to tackle this issue.

Thank you,

Regards,

Thomas

2

Steps to Reproduce
No repro steps found for now.

3

Hello!

I searched our internal DB and could only find 1 fix that should already be in version 5.6.0 (CL#42412670). Do you have more details than those stacks?

Regards,

Martin

4

Hi,

This look like a CPU crash and doesn’t appear related to the GPU. It’s likely a race condition that results in memory corruption or an overwrite.

I’m not sure what you get from a Sentry report but the state of all threads could be useful. The info usually makes it in CrashContext.runtime-xml but the stacks would not be resolved which wouldn’t help…

Martin

5

The upload failed. Can you try again?

6

The stacks are inconclusive as none is showing activity that could indicate memory overwrite. You should review the custom classes in your project and make sure that you are not missing a UPROPERTY tag on UObject pointers. Untagged references are often linked to this type of problems.

7

Hello Martin!

It reproduced on various GPUs:

NVIDIA GeForce RTX 4090, NVIDIA GeForce RTX 4070 Ti,NVIDIA GeForce RTX 4070 SUPER, NVIDIA GeForce RTX 4070, NVIDIA GeForce RTX 3080, NVIDIA GeForce RTX 3060, NVIDIA GeForce RTX 4080 SUPER, NVIDIA GeForce RTX 4070 Ti SUPER, NVIDIA GeForce RTX 4060 Ti, NVIDIA GeForce RTX 3060 Ti, Intel(R) UHD Graphics 770, AMD Radeon RX 7900 XTX, AMD Radeon RX 7900 XT

Driver versions (not sure if relevant)

560.94, 576.40, 581.08, 591.44, 591.74, 591.86, 595.79, 32.0.101.7082, AMD Software: Adrenalin Edition 23.9.3, AMD Software: Adrenalin Edition 26.2.2

From the custom events we have, it seems to happen during level loading, but from the callstack it should be expected.

We will try soon to make some autotest with -rhivalidation for the various gpu crashes we also got, do you think it could help here ?

If there’s anything specific you are looking for from our Sentry report, let me know.

Regards,

Thomas

8

Here’s the state of the threads.

9

Yes, apparently we cannot send .txt and .zip here :frowning: (I tried 3-4 times)

Here’s a gist: Threads · GitHub

Regards,

Thomas

10

Interesting, we will check this, thank you.