Huge Memory Calculation Regression (Missing Infos, Obfuscated texts, Missing assets in calculation)

General Issues and Bug Reporting
, , ,
1

Summary

This has been basically since the new memory calculation system was introduced (Sorry for not reporting earlier :))

New memory calculation has a few issues that make it really hard to optimize:

  1. Memory calculation does not sum up correctly
  • The memory in the calculated final result in this map is around 97k / 100k
  • Summing up all individual parts does not sum up to the correct number so alot of assets are missing in the calculation output

This means either the memory calculation is missing some assets that are not correctly shown, is wrong or does not show all assets.
When optimizing huge open world islands it is really important to know every asset taking up memory.

  1. Fortnite Assets used to be naming the correct assets (Marked in the screenshot in red)
    Now after the new calculation it only says “Fortnite asset”. This is extremely hard to optimize since if you use a bunch or alot of different fortnite assets you basically have no idea which asests caused your memory to increase

  2. Huge issues with spline assets (Example spline asset in marked in purple in screenshot)

  • Spline assets (made using the landscape spline tool) now show up obfuscated as random letters
  • Spline assets can increase the memory by alot (Sometimes up to 30k memory of 100k memory )
  • Spline assets somewhere on the map even if spatially loaded seem to increase the memory everywhere

Please select what you are reporting on:

Creative

What Type of Bug are you experiencing?

Assets

Steps to Reproduce

  1. Open a big project using fornite assets and custom ones + splines
  2. Calculate Memory
  3. Look at memory calculation

Expected Result

Memory calculation result shows all things taking up memory and exactly tells what assets are taking it up (Used to work correctly before memory calculation optimization)

Observed Result

Memory calculation is missing lots of infos, some assets say uknown and basically all fortnite assets you have no idea what the asset is

Platform(s)

windows

Island Code

0510-0384-5646

Upload an image

image (1).webp
image (1).webp2329×1095 959 KB

image (2).webp
image (2).webp1402×421 6.19 KB

image.webp
image.webp1644×1043 24.7 KB

2

image
image1402×421 9.61 KB

For example with this screenshot the final result says 97k but summing up this is not even close to 97k

4

Found a screenshot from the old system which was clean and very descriptive. New system is a mess and the only good thing about it is the extra memory we ve gotten, but knowing this system all these years tells me that this extra memory is not good either as it creates issues on various platforms. So what is the point in going through all this massive process to optimize if even the thermo is labeled wrong. Also I previously mentioned spatial profiler has been useless overall to help with any of this (please educate me if I am wrong, but I ve asked others and they said the same).

What happened with memory, why has it gotten worse than the UEFN release?

Discord_IX0SMcrmLl
Discord_IX0SMcrmLl1190×826 145 KB

1 Like
5

The issues with spline assets is one of the most important issues to solve for us. Bumped our memory by 30-40K on Drive Nation.

2 Likes
6

Hi @chrisp_games ,

Thanks for your feedback about the memory calculation. Let me respond to your issues:

  1. Memory calculation does not sum up correctly

  • The memory in the calculated final result in this map is around 97k / 100k

  • Summing up all individual parts does not sum up to the correct number so alot of assets are missing in the calculation output

The report contains the top resources referenced by your project, up to 100 resources. Summing up all of these parts will not match the total value shown by the in-game thermometer if you are referencing over 100 resources (which is likely).

  1. Fortnite Assets used to be naming the correct assets (Marked in the screenshot in red)
    Now after the new calculation it only says “Fortnite asset”. This is extremely hard to optimize since if you use a bunch or alot of different fortnite assets you basically have no idea which asests caused your memory to increase

The new memory calculation report lists memory usage via a “bottom-up” approach (i.e. the list of assets being referenced by all objects), vs the “top-down” approach used by the old system (i.e. what is the total size of all the assets each object is referencing).

There are pros and cons to both approaches, but we decided to go with the bottom-up approach to list the problematic assets, as with the old system an object A could be using X amount of memory, but it wasn’t clear which element of the object was contributing to its high memory use.

The main downside you have identified is that for internal Fortnite assets, it isn’t clear what they are aside from the asset type (if we can identify it). In practice, you can’t make any changes to these assets to reduce their memory use, so knowing the name of it is irrelevant. However, we do provide the “Browse to Actor” link which selects an actor in the world which is referencing the resource. This can be used to get a sense of which type of actor is referencing the resource, and you can optimize from there.

  1. Huge issues with spline assets (Example spline asset in marked in purple in screenshot)

  • Spline assets (made using the landscape spline tool) now show up obfuscated as random letters

  • Spline assets can increase the memory by alot (Sometimes up to 30k memory of 100k memory )

  • Spline assets somewhere on the map even if spatially loaded seem to increase the memory everywhere

The assets you’ve identified with the random letters are cooked World Partition cells/levels, rather than spline assets specifically. We do need to take steps to improve the names shown for these WP cells, so thanks for calling that out.

In terms of the spline meshes affect on memory use, it’s possible that it’s due to the spline meshes generating extra vertex and collision data for rendering and physics. However, I’m not a specialist in this area. You might want to look at the level of detail you’re using for these meshes, with regards to rendering and physics.

There have been some small improvements made to the UEFN Memory Calculation report which are scheduled to land in the near future, which should help with some of the cases of unknown assets showing up. Otherwise, I’d recommend looking at the troubleshooting page in the documentation below on information that might be useful to reduce your island’s memory calculation:

Thanks,

Nick

7

Hey Nick, appreciate the breakdown and confirmation about splines, we ourselves are running into a massive block with mem calc results trying to optimize a large project, and there are certain things I want to flag as well:

  1. The old system showcased cases where you have two props that are the same mesh but have different Texture Data on them, which made their instances cost less. You cant see that anymore with the new system.
  2. Old version had a detailed breakdown on what each device meant in the level, now we don’t really know what are devices and verse doing.
  3. On knowing the FN asset name, I would love to know the name because a) I can find another similar asset that uses less memory (trees and furniture are good examples), or b) recreate it as an SM on my own.
  4. We no longer know things like instance costs to in-game memory, or how far a prop affects memory. These are crucial things to optimize with the thermo in mind.
  5. Current system doesnt “bundle” instances right? That’s why the top 100 aggregated are so off from what is the actual memory. Either way, its not useful for optimization if it doesnt paint the full picture.

Overall, the problem I have is that we are trying to apply a custom workflow on an existing workflow that relies on an obscure number which is now even more obscure.

8

I created this earlier

Before this post i believe

I agree 100% this is not at all practical to use now.

As a creator i now have no idea what anything is when its a grouped thing. Everything is its own item which just confuses the whole situation.

I feel the team has done a lot of work on the mem calc itself which is great and faster ect but i feel this part of the system the info side should have had some feedback from the community before drastic change was implemented.

I don’t wanna be negative at the changes but i cant think of 1 good reason to use it anymore i might as well ignore it and just create an island that’s within the mem thermo gauge and forget about optimizing on a per asset bases.

The 2nd big change that most creators dont like is the fact the thermo gauge is now not live. Before we could drag say a tree in the world itd increase we could then compare to another tree ect now you have to push changes which could be anything upto 10 minutes to check the mem value of the tree.

Just wanted to share my views

9

Hey Nick, really appreciate the quick aswer !

Here are a few more things I was thinking about

  • In the provided screenshot it says 97k of 100k memory. If I sum up the top resoruces it is around 60k memory. Since this is sorted from top to bottom for most demanding resources and the bottom one is just 100 memory I am still missing around 37k memory. Even if it only shows the top resources if the bottom one of the top resources is just 100 could this still mean something is missing here?

  • The spline one I kinda understand but I am not quite sure why in my maps this usually affects the whole island instead of just the local area of the loaded world partition cells. Also I have enabled spatial loading on splines to decouple them from the landscape proxy but it does not affect memory at all. Would not this decrease memory due to the spline only rendered up to certain distances. I also checked the spline mesh and it very low poly and uses simple as complex for physics

  • Also if the report is already calculating memory on the whole island anyways is there any way to print out or get a file that has all the memory calculation results not just the top ones. On big open world maps it is really necessary to know every asset that is taking up memory. Also would love to get this to assure everything sums up correctly

Thank you already for the quick answers! Also really looking forward to the upcoming memory calc improvements! :slight_smile:

10

(post deleted by author)

11

Hi @Wertandrew ,

You raise some valid points, and we’re tracking the gaps in functionality between the old and new solutions. However, the speed and determinism of the new solution did require that we needed to make some tradeoffs, but there’s always scope for improvements and your feedback is appreciated.

It should be noted that the report doesn’t hide anything with regards to the value the thermometer is presenting in game. Instance costs for objects contribute to the overall cost of the level they are contained in - we don’t have a way to isolate these with the new system, unlike with the old system. However, the cost of each instance is ultimately accounted for in the thermometer and memory calculation. This is usually a very small cost in comparison to the cost of the assets referenced though.

Thanks,

Nick

1 Like
12

Hi @AtAshTag ,

I apologize for not seeing and responding to your post before. I appreciate your feedback, and want to assure you that we take community feedback into consideration when developing updates to these systems. However, there are tradeoffs that we need to make when developing a solution.

Previously, we were seeing cases with the previous solution where the true memory cost of island content was being hidden behind actor names. For instance, you could have two or more objects sharing a common large texture, and it would be difficult to isolate where this large shared cost was coming from. The previous solution could also take a very long time to run on large islands.

The new solution addresses both of these issues to some degree, but you’re right to call out these issues as you see them, and we’ll take this information on board for future improvements in this area.

Thanks,

Nick

1 Like
13

Hi @chrisp_games ,

In the provided screenshot it says 97k of 100k memory. If I sum up the top resoruces it is around 60k memory. Since this is sorted from top to bottom for most demanding resources and the bottom one is just 100 memory I am still missing around 37k memory. Even if it only shows the top resources if the bottom one of the top resources is just 100 could this still mean something is missing here?

In this case, it’s like that there’s a long tail of small resources which are being referenced, which are contributing to your overall memory costs. This is one of the areas where the top 100 resources report can be improved.

The spline one I kinda understand but I am not quite sure why in my maps this usually affects the whole island instead of just the local area of the loaded world partition cells. Also I have enabled spatial loading on splines to decouple them from the landscape proxy but it does not affect memory at all. Would not this decrease memory due to the spline only rendered up to certain distances. I also checked the spline mesh and it very low poly and uses simple as complex for physics

In this case, I would make sure that other things in your world are also set to be Spatially Loaded, such as the Landscape Proxy Actors themselves - I have seen cases where they have been disabled before.

Also if the report is already calculating memory on the whole island anyways is there any way to print out or get a file that has all the memory calculation results not just the top ones. On big open world maps it is really necessary to know every asset that is taking up memory. Also would love to get this to assure everything sums up correctly

At this time there isn’t a way to get this, but it’s something I’d like to offer at some point in the future.

Thanks,

Nick

15

FORT-1025460’s status has changed to ‘Ready for QA’. A member of the QA department is investigating the issue.

16

Hey

@nick.edwards.epic

I’m struggling to understand something with the mem calc.

I have 1 texture which is 2048x2048

image
image553×209 14.1 KB

This is an imported texture yet this takes 5,593 in game memory ?

Isn’t the imported assets suppose to take project size not the thermal gauge memory or have i gotten mixed up ?

I have re done the texture so its much lower now but feel this is a good one to show.

Lastly if you have an asset with the same texture as another type of asset would each asset hold a mem value for the texture as a new value or should then start to have like an instance value as you already have one in the map. Abit like how a device works you have one then its a big chunk then any other is much less ?

17

Is this the reason why trying to go into “Tools→Audit→Statistics” and selecting “Actor Memory Stats“ in dropdown just straight up crashes the editor now?

18

They are compressed and included in the project which reflects on the project size. When they are used (for rendering), they are uncompressed and loaded into memory which reflects on the thermal gauge.

Fortnite assets are already included with the installation of Fortnite, so one doesn’t have to include them with their project. However, when used, they still need to be loaded which affects memory usage.