We’ve noticed a high amount of memory attributed to Nanite.PageUploadBuffer during long walkthroughs. This shows up when running memory investigations in Insights memory traces, whether it’s Active Allocs or Growth. The amount seems to increase all the time. I checked the CLs mentioned in this thread [Memory leaks from Nanite and DistanceFields buffers when changing the [Content removed] but they didn’t fix the issue. I ended up doing some digging and came to the conclusion that this is an issue with tracing. Aside from that, I found no evidence of an actual memory leak. The culprit, in my opinion, is the trace realloc below, in FD3D12DynamicRHI::LockBuffer. There is no matching alloc/free. In addition, when I commented it out Nanite.PageUploadBuffer allocations under “Untagged” disappered completely. The remaining ones seem reasonable, I guess?
I’d appreciate it if you can confirm my conclusion or work on a solution if I’m wrong.
Note: The allocs in the screen shots are of Nanite.PageUploadBuffer exclusively, set in a filter. We’ve noticed the behavior on both PS5 and XSX, and haven’t checked other platforms.
Apologies for the delay, this does look plausible, but it isn’t clear yet if this is something we’re seeing in latest. I hope to get some time soon to set up a test case and see if we’re also seeing this behavior in vanilla with, say City sample or similar.
I’ve been attempting to replicate the behavior you’re seeing on PC in latest using level sequence to fly over nanite geo repeatedly and switching between that level and an empty level.
[Image Removed]The LockBuffer allocations don’t appear in Untracked for me. It’s possible this issue has been fixed, or I’m not reproducing it the same as you are. On PC I had to change some CVars to try and simulate the environment by enabling dynamic page uploading and constraining resources:
; Enable dynamic page upload buffer on PC
r.Nanite.Streaming.DynamicPageUploadBuffer=1
r.Nanite.Streaming.NumInitialRootPages=512
r.Nanite.Streaming.DynamicallyGrowAllocations=0
r.Nanite.Streaming.StreamingPoolSize=16
I’ll reach out to colleagues in case they have seen this kind of behavior also, but if you can share a project for reproducing this issue in vanilla UE that can help us debug the issue.
My colleagues suggested it’s possible what you might be seeing is a classification issue. We fixed something like this a while ago in CL#41015041 (d91257) Add support to move allocation tracing (LLM, Insights) from one tag to another.
Since 37914651, any buffer allocated through FRDGBufferPool::CreateBuffer has the tag RHIMisc/BufferPool. This makes sense for most buffers, as their lifetime is short, and they are shared by multiple systems.
Some of them, like the Nanite buffers, are exclusive and persistent, so we want to put them in the nanite category.
The main chunk of memory is allocated within the RHI and is different per platform: add UpdateAllocationTagsForPooledBuffer / FRHIBuffer::UpdateAllocationTags so that every RHI may deal with its specificities.
Add support for reserved resource support since the move may happen before or after the commit.
Fix Insights memory tracing tagging for reserved resources because LLM_REALLOC_SCOPE is not rebasing the tags from Insights: UE_MEMSCOPE_PTR is needed instead.
- I verified, we do have CL#41015041 (we’re on 5.7)
- I checked on PC. I don’t see the issue, Nanite.PageUploadBuffer is only reported under RHIMisc/BufferPool and Nanite tags. Here are the CVar values on PC
- I double checked the CVar values on XSX, r.Nanite.Streaming.DynamicPageUploadBuffer is set to 1, that comes from Engine\Platforms\XSX\Config\XSXEngine.ini. I tested setting it to 0 which cleared Nanite.PageUploadBuffer allocs under Untagged
- I double checked on PS5. We don’t have the issue actually. The CVar is set to 0 and setting it to 1 doesn’t change how Nanite.PageUploadBuffer allocs are traced
- I circled back to PC and tested with the CVar set to 1. Got the same result as with it set to 0
So basically, this happens on XSX only, with the CVar values below. I don’t think it’s a Nanite issue though, I think it happens anytime that branch of the code is taken, which could happen with other systems but at a much smaller scale (like Niagara or Slate …). Basically whenever Lock Mode is set to RLM_WriteOnly, there is a chance to fall into this pattern
r.Nanite.Streaming.DynamicPageUploadBuffer=1
r.Nanite.Streaming.NumInitialRootPages=2048 // (not important as r.Nanite.Streaming.Debug.ReservedResourceIgnoreInitialRootAllocation=1)
r.Nanite.Streaming.DynamicallyGrowAllocations=1
r.Nanite.Streaming.StreamingPoolSize=160
Thanks, that does provide some useful information. We may need to wrap the MemoryTrace_ReallocFree/MemoryTrace_ReallocAlloc for XSX conditions like:
#if UE_MEMORY_TRACE_ENABLED && (D3D12RHI_PLATFORM_LOW_LEVEL_RES_TRACK == 0)
// This memory trace happens before RenameLDAChain so the old & new GPU addresses are correct.
// Only emit realloc when per-buffer VAs are traced here. When D3D12RHI_PLATFORM_LOW_LEVEL_RES_TRACK=1
// the backing heap is traced at allocation time so ReallocFree of a pooled upload VA is a no-op
// and ReallocAlloc would leak Untagged VideoMemory in Insights.
MemoryTrace_ReallocFree(Resource->ResourceLocation.GetGPUVirtualAddress(), EMemoryTraceRootHeap::VideoMemory);
MemoryTrace_ReallocAlloc(NewLocation.GetGPUVirtualAddress(), Resource->ResourceLocation.GetSize(), Resource->BufferAlignment, EMemoryTraceRootHeap::VideoMemory);
#endif
Thank you Alex! I partially integrated CL45505302 in order to get D3D12RHI_PLATFORM_LOW_LEVEL_RES_TRACK, as it didn’t make it to 5.7 release. That said, I could have used D3D12RHI_PLATFORM_HAS_UNIFIED_MEMORY instead. I tested your change on XSX and it indeed fixes the issue, I don’t see Nanite.PageUploadBuffer allocs under Untagged anymore. Thank you
Hey, apologies for jumping in on an existing thread but we see the exact same issue (on 5.7.3). It creates a lot of noise and we don’t know if they’re legitimate issues or not, as we are investigating our own memory issues.
