ZenServer MemroyLeak

Development Pipeline & Plugins
, ,
1

OS : Windows11

ZenVersion : 5.7.13

<br/>

When using ZenServer, we encountered an issue where memory kept increasing and eventually caused a crash when cached data from PC A was used on PC B. After investigating the cases where memory continuously increased, we confirmed that the problem occurred during serialization in the call stack below. In this situation, the NewNum value becomes incorrect, causing an extremely large number of elements to be added to the array, which leads to the memory increase.

<br/>

-----

Callstack:

<br/>

[Inline Frame] UnrealEditor-Core.dll!FMallocMimalloc::TryMalloc(unsigned __int64) Line 83 C++

UnrealEditor-Core.dll!FMallocMimalloc::Malloc(unsigned __int64 Size, unsigned int Alignment) Line 90 C++

UnrealEditor-Engine.dll!FMemory_MallocInline::__l2::<lambda_1>::operator()() Line 39 C++

[Inline Frame] UnrealEditor-Engine.dll!AutoRTFM::Open(const FMemory_MallocInline::__l2::<lambda_1> &) Line 891 C++

[Inline Frame] UnrealEditor-Engine.dll!FMemory_MallocInline(unsigned __int64) Line 25 C++

UnrealEditor-Engine.dll!operator new(unsigned __int64 Size) Line 9 C++

> UnrealEditor-Engine.dll!TIndirectArray<FStaticMeshLODResources,TSizedDefaultAllocator<32>>::Serialize(FArchive & Ar, UObject * Owner) Line 217 C++

UnrealEditor-Engine.dll!FStaticMeshRenderData::Serialize(FArchive & Ar, UStaticMesh * Owner, bool bCooked) Line 2536 C++

UnrealEditor-Engine.dll!FStaticMeshRenderData::Cache(const ITargetPlatform * TargetPlatform, UStaticMesh * Owner, const FStaticMeshLODSettings & LODSettings) Line 4220 C++

UnrealEditor-Engine.dll!UStaticMesh::CacheDerivedData() Line 7104 C++

UnrealEditor-Engine.dll!UStaticMesh::ExecutePostLoadInternal(FStaticMeshPostLoadContext & Context) Line 7874 C++

UnrealEditor-Engine.dll!FStaticMeshAsyncBuildWorker::DoWork() Line 327 C++

UnrealEditor-Engine.dll!FAsyncTaskBase::DoWork() Line 289 C++

UnrealEditor-Engine.dll!FAsyncTaskBase::DoThreadedWork() Line 313 C++

UnrealEditor-Core.dll!FQueuedThreadPoolWrapper::FScheduledWork::DoThreadedWork() Line 110 C++

[Attachment Removed]

2

Hi,

I’m sorry for this issue. I don’t believe it is a memory leak, but I do think it is corrupted/invalid DDC data being provided by zenserver. Post 5.7 I added more validation to make sure that if zenserver sends bad data, we are more aggressive about detecting and rejecting the data instead of trying to deserialize it. This was done in UE5/Main changelist 48523454 and the following github commit:

https://github.com/EpicGames/UnrealEngine/commit/f3141f5b8d3574f362df0abace29459bb66c6e3c

Could you try taking that change and seeing if it addresses the issue you’re encountering? I’d also recommend taking the newest release of zenserver (5.7.19 at the time of writing) to make sure you have the latest fixes for the server also. Taking newer releases of zenserver is possible without updating your engine release.

[Attachment Removed]

3

Thank you. After upgrading to ZenServer 5.7.18 two months ago, the issue has not been reproduced so far. If it happens again, I will apply the commit you suggested and test it.

[Attachment Removed]