I want to learn more about how Vram is used and texture/object optimizations.

Development Rendering
1

I have a** 2.5d platformer** with the camera traveling linearly from left to right until end of level is reached. In this level I need to be able to load high res 4K sometimes 8K textures on backgrounds and objects.

Even though the level is not infinitely big I do run the risk of running out of video memory on lower end cards to keep all those high res textures all loaded at level start which in my case are the largest memory consuming part of the game.

I have been reading quite a bit about texture or object streaming and mipmapping, I suppose the only two methods currently available to assess and optimize (I am excluding the newly implemented virtual texturing for the time being). But i’m still a bit in the dark here as how the workflow logic behind them is working, how the hardware is communicating with the engine and how best to optimize all this.

I know this is a big topic to cover but I’ve been browsing around in forums and many new devs ask this question and end up with bits and pieces of information. I hope some of you can point out a good chunk of info in this post to have it less spread out perhaps for other people who might be interested other than myself.

I will lay out what I get out of the two methods so far and hope someone can clarify them further.

Mipmapping:

I understand that mipmapping is an autogenerated process by the engine to split your texture into different resolutions (512, 256 etc…) and switch between them for either better bandwidth or to avoid flicker problems on distant objects, here are my questions for this method:

1 - What if I don’t have any distant objects so the flicker question is out the window for me.

2 - The texture is supposedly already being loaded inside Vram and then split up? and by doing so according to some users this adds 33% more consumption on video card memory, how is this helping?

3 - What is bandwidth and how is it being used exactly?

4 - Mipmapping sometimes causes artifacts as it shows the textures being loaded from low res to high, which is undesirable.

5 - What If I don’t want to use mipmapping and would like to keep textures as they are since only one or two sections of the levels is viewed at one time.

Perhaps for a 2.5d platformer type games texture streaming is better since (correct me if I’m wrong) I don’t see how mipmapping helps in this case?

Texture streaming:

From my understanding Texture streaming loads textures similar to level or object streaming inside ue4. If the camera is not looking then it doesn’t need to be shown on that specific level section.

My questions:

1 - Are the textures stored inside Vram or are they being loaded from Hard drive onto Vram when called for? I suppose the latter?

2 - I read the docs, but can someone better explain to me how this whole thing is working?

3 - During object streaming in my levels I’ve been noticing a small glitch during gameplay during the trigger when loading them. Is this something I can also expect to happen when trying to stream textures?

4 - Any recommended good tutorials on the usage of this subject.

Object and level streaming during game play:

Ultimately is it not better to keep textures as they are without mip maps and texture streaming and instead simply load them with the object/level streaming instead, in a case of the type of game I am referring to? Since the objects already include the textures so why separate the loading process when in any case i may be streaming object load during gameplay?

But then again someone may be able to tell me the pros and cons of all this. One of which I already mentioned above there are glitches and game pauses happening on every object load/stream during gameplay, perhaps I need to spread out and make more micro loadings on a larger span of time?

How would you best approach optimizing levels for this type of game?

Thank you in advance!

2

The flicker problem is caused when you have a really high res texture which is rendered onto a small surface. It’s pretty much an aliasing problem. Imagine you have a 1024x1024 texture and you are rendering it onto a 32x32 quad on the screen. The question for the GPU is, what samples from the texture should be sampled for the 32x32 quad? The GPU will generally do pixel interpolation on the texture. The flickering starts to happen when the size of the 32x32 quad changes and the interpolation results change as well.

Mipmapping 2:
Yes, adding an extra 33% memory usage is a small price to pay. Mipmapped images are pre-interpolated for various resolutions, presuming that the textures are saved as powers of 2. The GPU doesn’t need to do as much rasterizer work sampling source pixels for smaller triangles. That means it can draw the triangle faster!

Mipmapping 3:
Bandwidth is the act of moving data from main memory to GPU memory. Every frame, you have to move data from memory to the GPU to draw things. This includes textures, shaders, meshes, and anything else that needs to be rendered.

Mimapping 4:
You can treat mipmaps sort of like an LOD. What you’re describing is popping from one LOD to another. In an ideal world if you have perfect LOD transitions, the popping is unnoticeable. If its noticable, then the LOD switch is happening too soon. This is one of those things you may just need to tweak manually to get best results.

Mipmapping 5:
You can manually turn off mipmapping on your texture settings.

Texture Streaming 1:
Textures are temporarily stored in VRAM, just for the draw call. Generally, the textures are loaded from the hard drive into main memory, and then moved from main memory into VRAM in order to make the draw call. After the draw call is done, the VRAM is dumped and prepared for the next draw call. You will have hundreds to thousands of draw calls per frame rendered. The main hitch to rendering is when a texture is not in main memory, but is required for the draw call and needs to be loaded from disk. Disk I/O is Sloooooooow… Usually an engine tries to figure out how to minimize disk IO by being smart about when to load textures from disk into main memory and then into VRAM.

Texture Streaming 3:
The hitching you’re seeing may not necessarily be caused by texture streaming.

Generally speaking, if you really want the nitty gritty on what’s happening under the hood, you need to look at the GPU analyzer and look at each draw call. It’s going to be a pretty tedious process to look at all of the DirectX calls and understand what they mean.

3

The flicker problem is caused when you have a really high res texture which is rendered onto a small surface. It’s pretty much an aliasing problem. Imagine you have a 1024x1024 texture and you are rendering it onto a 32x32 quad on the screen. The question for the GPU is, what samples from the texture should be sampled for the 32x32 quad? The GPU will generally do pixel interpolation on the texture. The flickering starts to happen when the size of the 32x32 quad changes and the interpolation results change as well.

Mipmapping 2:
Yes, adding an extra 33% memory usage is a small price to pay. Mipmapped images are pre-interpolated for various resolutions, presuming that the textures are saved as powers of 2. The GPU doesn’t need to do as much rasterizer work sampling source pixels for smaller triangles. That means it can draw the triangle faster!

Mipmapping 3:
Bandwidth is the act of moving data from main memory to GPU memory. Every frame, you have to move data from memory to the GPU to draw things. This includes textures, shaders, meshes, and anything else that needs to be rendered.

Mimapping 4:
You can treat mipmaps sort of like an LOD. What you’re describing is popping from one LOD to another. In an ideal world if you have perfect LOD transitions, the popping is unnoticeable. If its noticable, then the LOD switch is happening too soon. This is one of those things you may just need to tweak manually to get best results.

Mipmapping 5:
You can manually turn off mipmapping on your texture settings.

Texture Streaming 1:
Textures are temporarily stored in VRAM, just for the draw call. Generally, the textures are loaded from the hard drive into main memory, and then moved from main memory into VRAM in order to make the draw call. After the draw call is done, the VRAM is dumped and prepared for the next draw call. You will have hundreds to thousands of draw calls per frame rendered. The main hitch to rendering is when a texture is not in main memory, but is required for the draw call and needs to be loaded from disk. Disk I/O is Sloooooooow… Usually an engine tries to figure out how to minimize disk IO by being smart about when to load textures from disk into main memory and then into VRAM.

Texture Streaming 3:
The hitching you’re seeing may not necessarily be caused by texture streaming.

Generally speaking, if you really want the nitty gritty on what’s happening under the hood, you need to look at the GPU analyzer and look at each draw call. It’s going to be a pretty tedious process to look at all of the DirectX calls and understand what they mean.
[/QUOTE]

Thanks for your reply.

4

Even if your camera is always looking at a perpendicular flat plane at constant distance, your screen resolution is still a variable, thus even here mip maps are still applicable.

33% inflation is always worth when matched against aliasing and cache efficiency.

It is amount of data over time, that needs to make it from video memory to a texture units.

It is side effects of streaming system, not mipmapping.

It is arguably only justified in a case of flat plane, perpendicular to camera and only of at lowest playable resolution, you will not be getting more than 1 texel per pixel.

Latter.

Whole thing analyzes material and how it uses textures with respect to UVs, assumes a dependency between UVs and object’s size, and given this data and a a bit of logic, attempts to keep in memory only mips, that are relevant.

You can expect streaming system to fail streaming required mips in due time, as well as not dropping unneeded mips promptly, but all of that is manageable with streaming settings.

5

@Deathrey Thanks for pitching in!

So I assume what you are saying is that if the overall game resolution is decreased by the player in the options UE automatically switches to low resolution mips to further compensate?

So if an object is off screen UE assigns the lowest possible mip size to that object until the object is visible to the camera? Also does texture streaming require you to have mips assigned to all your textures ?

Overall what I seem to be getting out of all this is the advice of keeping the mipmaps on all textures as they come by default in Unreal and just play about texture streaming in the level?

Another question I had in my original post which is very relevant to my project:

I am loading in assets inside the level using triggers in Sequencer. Basically the assets are not “Visible” and then after triggered they turn on “Visibility”. I did this to sort of try and work out if it helps with performance on a large level when it comes to object and texture memory being loaded inside Vram. Is this another way I could “Stream in assets” or is this a wrong approach. Since these assets are being “Loaded in” with everything including textures and materials assigned I was a little confused as to what benefits other methods including texture streaming would have vs just loading entire assets in incrementally that are off camera?