We have around 500 textures that are used as thumbnails in our project.
They are defined as:
UPROPERTY(EditDefaultsOnly, BlueprintReadOnly, Category = Info, meta=(AssetBundles="B"))
TSoftObjectPtr<UTexture2D> Thumbnail;
We noticed when using the resource viewer tool, that all of our item thumbnails are loaded into the VRAM at once.
We could not find out why they would be loaded since they are not hard-referenced or displayed anywhere.
The only thing we are doing is loading a PrimaryDataAsset with a specific bundle (Bundle A). The texture itself is only in BundleB which should never be loaded.
But for some reason as soon Bundle A is loaded in the pie session, these textures pop up in the VRAM.
Im a bit over my head here, so my questions would be:
- When are textures loaded into the VRAM and what is the best way to debug what is actually loading them?
- What are the best approaches to handle Texture loading for thumbnails of huge lists (eg. inventory that could have 500 different items) ?
- I also heard that virtual textures should not be used for UI. Is this still accurate or can we try to use that here?
[Attachment Removed]
Steps to Reproduce
not really a bug report but more of a question.
[Attachment Removed]
Hi,
I’ll start by answering your questions:
When are textures loaded into the VRAM and what is the best way to debug what is actually loading them?
It’ll probably be a bit tricky to get a callstack for where exactly a given texture is loaded to VRAM. You could try dropping a breakpoint in FTexture2DResource::InitRHI, though I imagine there will be quite a few calls to that so it may be tough to catch one of the thumbnail textures you’re looking for. It’s also worth throwing a breakpoint on FSoftObjectPtr::LoadSynchronous to see if that’s being called somewhere.
What are the best approaches to handle Texture loading for thumbnails of huge lists (eg. inventory that could have 500 different items) ?
You’ll typically want to use some sort of virtualized widget, such as a TileView. The widget will automatically manage a pool of widgets and assign them to the different inventory items as you scroll, so each inventory item would be represented by a UObject containing a reference to the texture (and whatever other data you need to represent the item) and you’d handle assigning it to the image widget in OnListItemObjectSet.
I also heard that virtual textures should not be used for UI. Is this still accurate or can we try to use that here?
It should be fine to use virtual textures for UI as of version 5.6, support was added in CL#40218055.
Returning to your original problem, does anything show up in the reference viewer for one of those thumbnail textures? I’d expect to see those assets pulled in if something from Bundle A leads to the soft object pointer being resolved, which could happen in a constructor or something similar. If you’re able to share an example of how the thumbnail textures are currently used, we can see if anything jumps out.
Best,
Cody
[Attachment Removed]
Hi,
Thanks for the answer.
Regarding that i have a few questions.:
“You’ll typically want to use some sort of virtualized widget, such as a TileView. The widget will automatically manage a pool of widgets and assign them to the different inventory items as you scroll, so each inventory item would be represented by a UObject containing a reference to the texture (and whatever other data you need to represent the item) and you’d handle assigning it to the image widget in OnListItemObjectSet.”
How would the setup look like?
I would have a Viewmodel object that holds a pointer to the texture. Would this texture not already be loaded in the VRAM because its hard referenced by this object?
Then i would not really have an advantage
If i would soft ref it i would need to load it async on “OnListItemObjectSet”.
I have not tried it yet but i would imagine that you get a noticable delay while scrolling through the items in this case.
Is this how we should set it up or am i missing something?
Thanks in advance 
[Attachment Removed]
We tried using Virtual Textures for UI icons. However, when the UI is opened in-game for the first time, the icons render as solid colors for a single frame before resolving to the correct textures. Is this expected behavior, and is there a way to prevent or pre-warm this to avoid the one-frame artifact?
[Image Removed]
Does this lazy loading you mentioned only make sense to be used with soft refs or does it have anything to do with virtual textures as well?
[Attachment Removed]
Hi,
The lazy image widget would only work with soft references; we haven’t done much experimentation on using virtual textures with UI yet, though it’s something we may be exploring soon. For the texture pop, I wasn’t able to reproduce this (albeit in a much simpler setup), though this is in line with the more general pitfalls of using virtual textures. Those textures won’t be fetched until they’re first painted by the widget, so there’s quite a bit of work being done on the first frame as we pull all of these different textures at once.
There are a few cvars described here that might be helpful, it may be worth doing some testing to see if any of them prevent the icon pop. MaxUploadsPerFrame may be relevant here since you’re pulling in many icons at once. The alternative would be to try and paint the widget before it’s actually visible (i.e. under a loading screen), but I’m not sure how feasible that is since you’re working with an inventory screen and won’t necessarily know which textures to preload. A possible workaround could be for your icon widget to set it’s opacity to 0 for a frame, as that will cause the widget to paint without actually rendering anything.
Best,
Cody
[Attachment Removed]
Hi,
Thanks for your answer.
Unfortunatly setting the opacity or alpha to 0 does not help with this issue. Seems like they are not rendered / streamed in when we do this.
We also tried the MaxUploadsPerFrame cvar with no change.
https://drive.google.com/file/d/1knhesY13b2Kwibr\_sL\_yVDV3fjb7jJCE/view
I attached a repo here where its easy to see / reproduce this problem. This only happens the first time its made visible, so make sure to test in standalone.
Its also more appearent when limiting the frame rate to something like 20.
Then you can see it “pop in” for multiple frames.
Do you have any other tips what we could do to limit VRAM usage while still maintaining responsive and fast menus?
[Attachment Removed]
Hi,
Thanks for sending that sample project. We did a bit of digging, there’s a 1-pixel fallback image that you’re seeing here before the texture can be streamed in, set in FVirtualTextureDataBuilder::BuildLayerBlocks:
OutData.LayerFallbackColors[LayerIndex] = OnePixelImage.AsRGBA32F()[0];This fallback is set in FVirtualTexture2DResource::InitRHI:
ProducerDesc.LayerFallbackColor[LayerIndex] = VTData->LayerFallbackColors[LayerIndex];One possible fix here, if you’re able to make engine changes, would be to set that color’s alpha to 0 so that the fallback doesn’t show up at all. This would affect other VTs as well, but we’ll take a look at adding a setting that applies to specific texture groups so that you can exempt UI textures from using the fallback. I’ve put in a bug report to add that option (UE-361648).
Another hackier but less-intrusive workaround would be to set the opacity to something very low (i.e. 0.01) for a bit. You’re correct that opacity 0 doesn’t seem to result in the texture being streamed in, but an extremely low value will still bring it in without being noticeable.
If you don’t want to use VTs here and want to look into other strategies for reducing VRAM usage, I see that your colleague posted a question [Content removed] so I’ll share some thoughts on that post soon.
[Attachment Removed]
HI,
Thanks for your reply.
We tried setting opacity to 0.01, which is working. The only problem we have now is that we do not know when the texture is done using this fallback (when should we reset the opacity?).
Usually its around a frame but with low framerate this can take longer.
Is there any event to know when the texure is done using this fallback / is streamed in completely?
Otherwise we might try the engine modification, but were kinda scared it will cause problems with other things (eg. meshes changing their materials at runtime).
[Attachment Removed]
I’m not sure there’s an easy way to wait for the texture to be ready, there could be a lot of factors since we’re waiting on GPU operations that may not be in lockstep with the game thread, so you’d have to come up with a conservative estimate of how many frames you need to wait based on an expected worst case framerate. If you want to go with the engine change approach, you should be able to limit the risk if you can only make that change for textures in the UI texture group (or a custom texture group if you really want control over where it applies).
A good approach here might be to add a bool to FVirtualTexture2DResource to determine whether the fallback color should be used or not, and then set that flag (to skip the fallback) in the constructer while you can still access the owning texture’s LODGroup. Then, you could read that flag in InitRHI to clear out the fallback color.
[Attachment Removed]
Hi,
If the VM had a hard reference to the texture, I’d expect it to be loaded in RAM but not pushed to VRAM until it’s actually being rendered. The soft reference approach would have the downside you mention, that there may be a bit of delay as the widget is loaded. For Fortnite (which has a similar challenge with potentially huge inventories of cosmetics), we use UCommonLazyImage, which is a specialized version of UImage that handles lazily loading in the texture and displaying a loading indicator when it needs more time.
[Attachment Removed]