Hi there,
I think it’s a good idea to start with an overview of what virtual textures are actually giving you, and what the benefits, limitations, and drawbacks are.
The main advantage of virtual textures is their generally smaller memory footprint, and more granular streaming capabilities. With standard texture streaming, an entire mip of a texture is either streamed in or not. Furthermore, streaming is entirely handled by the CPU using only CPU available data (generally built when texture streaming is built). This means that you may be loading / keeping large amounts of texture data in memory that you don’t actually need.
Virtual textures use an entirely different streaming system, which is much more granular, and actually knows what mip, and even sections (tiles), of a texture the GPU actually needs. Virtual textures are streamed in tiles, which are usually smaller than the actual mip 0 texture. This means that only the tiles in the texture that are actually visible will be streamed in (default tile size is 128x128).
This all sounds great, however, it’s good to keep in mind that there are drawbacks to using virtual textures as well. Virtual textures primarily optimize memory usage, and might provide some performance wins if your game is constantly streaming in and out textures (trashing texture memory).
However, most of the time I would actually expect to see a general performance drop by switching everything to virtual textures. This is because there is extra overhead required for sampling and filtering a texture. Each texture sample needs to first sample the indirection page table, to check where in memory the actual texture tile is located before sampling from it. Extra instructions are also required to support texture filtering of virtual textures (anisotropy filtering support for virtual textures needs to be enabled as a separate option in project settings). Generally, texture filtering is not as robust on virtual textures. If you are using TAA or TSR (temporal anti-aliasing methods), you may not notice much quality difference. However, you may notice shimmering / aliasing artifacts if you are not using a temporal anti-aliasing method.
For landscapes, yes, you are right that these cannot use virtual textures when outputting to an RVT. This is likely due to the textures needing to be available immediately whenever an RVT renders. If the lower mips are not loaded in memory already, then the RVT tiles close to the camera will be rendered using the available high mips, resulting in blurry RVT output. The RVT tile then thinks that it is up to date, and the tile will stay blurry, even if the VT feedback system tells the CPU the next frame to load the required tiles for the source texture.
For parallax occlusion mapping (POM), I probably wouldn’t try to use virtual textures for the height map. POM is already quite expensive due to doing many texture samples in a loop per pixel. If you were to introduce virtual texture sampling for the height map as well, you would essentially be doubling the texture samples required per pixel for your parallax occlusion step. Technically you could probably get this to work with some effort. You would have to re-write the custom node (needed to support loops in a material) part of the POM material function to support virtual texture samples. You can technically sample from a virtual texture in a custom node, as is done for regular texture samples in the engine POM material function. If you still want to, you can take a look at the HLSL output of a material using a virtual texture sample to figure out how it does virtual texture samples in code.
In terms of import pipelines, you can enable auto virtual texture import in the project settings and specify a texture size above which all textures will get imported as virtual. Since VTs are mainly a memory optimization, it makes sense to keep small textures non-virtual to avoid the extra sampling overhead.
[Image Removed]
Flipbooks, and scrolling textures should technically still work, but I would check these in your project to see if you notice any blurry frames. Especially the first time the loop plays. Generally I probably wouldn’t use virtual textures for these anyway, as the use case seems like it would mostly be effects, and these tend to use lower resolution textures, which work better with the standard streaming system (lower performance overhead, already take up little memory). To get an idea of how flip books / scrolling textures might have issues, it might be useful to understand a bit about how the VT feedback system works. The feedback system randomly samples 1 pixel in every VT feedback tile size (default 16x16) pixels. When a texture sample happens in a shader which is writing to one of these pixels, it stores some extra data about the required mip level needed by that texture sample. Since this is a random sample, it’s possible it could take more than 1 frame to catch a required texture streaming update. So for panning / flip book textures, and even just general effect texture (for effects that appear and disappear suddenly), this can produce bad results, since the virtual texture feedback system might not be quick enough to keep up with sudden changes in texture sample locations / mips.
For testing / profiling, there unfortunately isn’t a nice way to just toggle VTs on / off at runtime and have them fall back to normal texture samples. The best way to test the performance / memory tradeoff is probably to package out a Test build of two versions of the same test level. One using virtual textures, and one not. Run each and compare the difference in performance (I recommend having a setup where the camera positions / orientation are fixed for this, or using a benchmark fly through camera so the tests are consistent).
My personal recommendation for using virtual textures, is that if you have a lot of unique, high resolution textures in your game, and / or you are struggling with texture memory budgets, then you should enable virtual textures on all your large textures (so long as you can afford the performance hit). If you are building for a mobile platform (or even if you’re not, but your game is more stylized and less reliant on lots of high res textures), then I would probably avoid virtual textures, and try to optimize texture memory using texture atlases and trim sheets. Specific texture usages may also warrant virtual textures, even if you decide to mostly use regular textures in your project. Some good examples are Hierarchical LOD (HLOD) textures. If you are using the merge setting with texture baking for your HLODs, then it will generate a unique texture for every HLOD in your level. This might add up to a significant amount of texture memory usage, even if your underlying meshes are efficiently re-using textures. Another is virtual texture light maps. This is an extra option you can enable in the project settings for baked lighting. Baked lighting often takes up a significant amount of texture memory, and a level texture memory footprint can be greatly reduced by using virtual textures here for the extra granularity. Though again, this comes at a performance cost.
Let me know if you have any follow up questions
Regards,
Lance Chaney
