Hello,
We’ve been running into editor crashes since we’ve turned on UseSimpleStreamableAssetManager in anticipation of testing FastGeo.
This has mostly been affecting our tech artist who’s working on our master materials.
They hit the following error
Fatal error: [File:D:\Horde\Sandbox\GT-Main-Code\Sync\Engine\Source\Runtime\Core\Private\Containers\LockFreeList.cpp] [Line: 49]
Consumed 1048576 lock free links; there are no more.
What we believe happens is this.
The users work with the main level opened and several material editors opened as well. She hits a crash on boot when the editor tries to restore the material editors.
The main level has about 120k primitives so the baseline is 120k SSAM primitive handles.
When the material editors are restored one or several FGlobalComponentRecreateRenderStateContext trigger a reregistration of every primitive into this table, this hits the level editor viewport and not just the material preview viewport so each pass adds 120k entries.
The table does not get a chance to drain between those triggers, by the time we open the 3rd editor, the number of passes is 9 which brings the total over the limit of ~1 million entries.
This setup is one way to repro the crash but we’ve hit this in other ways too. Another instance was during autosave, the render state was recreated because of the Thumbnail renders so the fragility is more general than just the editor boot flow.
Cheers,
Benjamin
[Attachment Removed]
Steps to Reproduce
In a project with s.StreamableAssets.UseSimpleStreamableAssetManager=1 set.
Have the editor set to open a massive level on startup.
Have the editor set to reopen previous assets on startup.
Open several material editors (including master materials and material functions) and restart the editor so that they restore.
You should observe a crash
[Attachment Removed]
Hi Benjamin, we recently doubled the size of that list for editor builds, ref CL 57770313
In SimpleStreamableAssetManagerHandle.h, change
static constexpr uint32 MaxSlots = 1u << 20; // 1M handles per tableto
static constexpr uint32 MaxSlots = 1u << (20 + WITH_EDITOR); // 1M handles per table (2M in editor)I’ll have that CL merged to the upcoming hotfix 5.8.4. In the meantime, changing it on your side should unblock you!
Joel
[Attachment Removed]
Well, even if that handle table had infinite capacity, we hit general memory limits elsewhere on large levels so it’s a balancing act. Since we recently hit this specific limit internally as well after optimizing other parts of the world code, we may end up switching to another less predictable but limitless capacity data structure.
Reopen if you hit this again and can’t get away with bumping the size, so we can prioritize. Thanks!
[Attachment Removed]
Thanks for the quick reply Joel, we’ll early integrate this change.
Are there plans for a more structural fix? This only kicks the can down the road, a bigger level or more assets opened simultaneously would still end up crashing.
This also triggers on autosave via the thumbnail render path. While we can be careful with how we open assets or even disable “reopen last”, the number of unsaved assets can easily explode beyond our control.
[Attachment Removed]