[IOS] [UE5.6.1] Physical memory regression on IOS after upgrading to Unreal 5.6.1

Development Platform & Builds Mobile
, ,
1

After upgrading from 5.5.4 to 5.6.1, we noticed a significant drop in available physical memory on iPhones. In particular, it’s pushing 4GB devices like the iPhone 11 into OOM kill territory. My two theories were it’s either from the engine upgrade, or from switching to IOS SDK, but after running regression tests on daily builds, the cutoff is clearly the engine upgrade.

I’ve attached the output from these tests, the Free/Max columns correspond to the same values in a CSV perf report. After the upgrade, spawning a crowd of bots is enough to terminate the app most of the time.

What’s frustrating is that memreports actually show a marginal improvement between the 5.5.4 build and 5.6.1, especially as we’ve identified some issues like uncompressed textures, unused UVs or excess LODs. But no amount of optimization is going to claw back ~400-500MB through no change of our own.

Worth noting that we’ve been using ANSI malloc on IOS due to stability issues on prior builds. Binned2 seems more stable with 5.6.1, but turning it on further increases physical memory use by ~200MB (the last 4 rows in the nightly table).

I’ve gotten pretty deep in the weeds running comparative Instruments memory traces. 5.6.1 does introduces a large VM allocation for the persistent linear allocator, but most of that remains uncommitted. I’ve confirmed by disabling it by setting PersistentAllocatorReserveSizeMB=0, without moving the needle at all. Actual memory usage on the persistent linear allocator seems to hover around ~15MB.

The only other thing that jumps out from comparing between the two versions is that 5.6.1 has many allocations for package loading callers like SetCustomVersions, FPakProcessedReadRequest, UpdateFromPackageFileSummary and SerializePackageFileSummary that don’t seem present in 5.5.4. But that makes little sense, as those functions did exist in 5.5.4, but there’s been enough changes to async loading that the callstacks must be different. (Instruments is an absolutely lousy tool for doing comparative profiling…)

I did briefly try Insights, but most LLM tags were Unknown and it was essentially unactionable.

Is there anything that might jump out as a likely source of memory regression on IOS in the 5.5-5.6 range? Apologies for a rather vague question without repro steps, but I’m running in circles and I’m at my wits’ end.

nightly-perftest-crowd.png(58.7 KB)
nightly-perftest.png(73.1 KB)

2

Hi Camille,

Between 5.5.4 and 5.6.1, significant changes on iOS were done in terms of memory mapping metal libs. This may be having a significant impact on iOS’s Free memory reporting, however, may not necessarily be indicative of more actual memory usage. Are you tracking this on Android as well?

Best regards.

3

Hi Camille,

We’ve had some reports of projects upgrading from 5.5 to 5.6 had theit Simulation -> Defauit Shape Complexity setting change to force all objects to complex. This did introduce a lot of additional memory usage at the time. Can you confirm whether your project was affected or not by this?

Best regards.

4

Hi Camille,

I’ve run this by the dmobile evelopment team and the believe that with UE 5.6.1 bringing along a fair amount of mmap’ing this may be artificially resuling in reporting of less memory. Have you noticed and degradation in stability as a result here?

Best regards.

5

Hi Camille,

Thank you for the additional information. I’ll circle back with dev.

In the meantime, does enabling the com.apple.developer.kernel.increased-memory-limit andextended-virtual-addressing entitlements get the app running again on iPhoneSE without being jetsam’d?

Best regards.

6

Hi Camille,

Did your pre 5.6 update enable occlusion testing and PSO caching? 5.6 enables those by default which if not enabled previously may be contributed to some memory usage increase.

Do you have any additional data since we last communicated?

Best regards.

7

Hi Camille,

Just checking in if you had more data. The team on this end is unaware of significant memory usage contributors in that version. Do you observe similar behaviour on templates created under 5.5 and upgraded to 5.6? If that is something you haven’t had the chance to look at, we will attempt in on our end in hopes of a repro that illustrates the issue.

Best regard.

8

The signal to noise ratio is a bit rough with some outliers, but it does seem to have had a noticeable jump as well.

Unfortunately there were no memreport snapshots in this automated pipeline back during the engine upgrade, I’ll have to sync back & rebuild to confirm but what is tracked by memreport actually looks improved (slightly less texture resource use).

9

trying this attachment again…

nightly-perftest-android.png(385 KB)

10

Alas, in a previous memory optimization pass, we had already made our project default to be DefaultShapeComplexity=CTF_UseSimpleAsComplex. And double checking in the content browser, only a handful of meshes have UseComplexAsSimple, though looking at those we can probably shave a couple of KB. Nothing to explain the larger delta on some devices, though--if it were collision shapes it would probably be consistent across all devices.

I took some time to rebuild a changelist closer to the 5.6.1 upgrade, to rule out content changes on our end vs the actual upgrade. Results are kind of confusing/all over the place, but most devices took a small to moderate memory hit from the upgrade. IOS devices lose a significant chunk of memory using Binned2 vs ANSI, whereas Android devices get a slight gain from Binned2... But this is all based on free/max physical memory readings, the actual memreports all come in nearly identical.

11

Yeah, the game was able to run on an iPhone SE 2nd Generation (3GB) without generally getting jetsam’d, and after 5.6.1 the iPhone 11 (4GB) is getting jetsam’d. The nightly perftest chart shows the max memory watermark doubling post-upgrade.

I figure most of this is from CL38689565 changing the memory footprint API, but the actual results with Xcode attached confirm that the process is indeed reporting more physical memory usage, enough to get killed in our heavier usage tests.

mmap changes seem like a decent lead, I didn’t find much that seemed to impact IOS between 5.5 and 5.6 though. Probably a long shot given how widespread mmap is, but are there any particular changelists I could revert just to A/B test?

12

We already have had those active, I suppose we’re running a pretty hefty title for mobile. :frowning:

13

We already had occlusion queries enabled, as well as PSO precaching. I did further enable occlusion feedback some time after the update, but that doesn’t seem to have had any impact on memory.

No new data on my part, I was running in circles trying to pinpoint the memory changes so I moved to other issues. The mmap changes seemed like the most solid lead, even though I agree that shouldn’t be counted as physical memory. I haven’t seen a singular CVar or subset of changes that I could back out to compare.