A copy paste from the documentation:
Streaming Virtual Texturing (SVT) is an alternative way to stream textures in your project from disk. SVT has several advantages—along with some disadvantages—when compared to existing mip-based in Unreal Engine.
Traditional mip-based texture streaming performs offline analysis of material UV usage and then at runtime decides which mip levels of a texture to load based on object visibility and distance. This process can be limiting because streaming data considered is the full texture mip levels.
When using high-resolution textures, loading a higher mip level of a texture can potentially have significant performance and memory overhead. Also, the CPU makes mip-based texture streaming decisions using CPU-based object visibility and culling.
Visibility is more conservative—meaning your system is more likely than not to load something—to avoid objects popping into view. So, if even a small part of the object is visible, the entire object is considered visible. The object loaded including any associated textures that may be required to stream in.
In contrast, the virtual texturing system only streams in parts of the textures that UE requires for it be visible. It does this by splitting all mip levels into tiles of a small, fixed size. The GPU determines which of the visible tiles are accessed by all visible pixels on the screen. This means that when UE considers an object to be visible, it’s communicated to the GPU which loads the required tiles into a GPU memory cache. No matter the size of the texture, the fixed tile size of the SVTs only considers the ones that are visible. Tile GPU computes visibility using standard depth buffers causing SVT requests to only happen for visible parts that affect pixels.
//
So when using virtual texturing you are requesting differing amounts of texture sizes depending on visibility and distance.
I would suggest you to look at some of the optimization talks on youtube.
Certain things dont always need 4k texture maps. depending on size of the object you can lower the texture size without losing much detail. Allot of the larger objects i personally use do use a 4k albedo map but the other textures like normal roughness i tend to lower to 2048-1024. Try lowering them and see if you could even tell the difference. Also texture packing is great, instead of having 5 different textures try packing them into 2-3! all of these things help greatly with performance and pool usage. 