I am trying to create a tool for our content folks, one step of which is to output our Landscape and Water Heightmaps to disk, for later post-processing. A lot of these are represented in-engine by Runtime Virtual Textures. I have noticed -- as is the expected nature of RVTs, I think -- that these textures are often lower resolution / quality than I would like when exporting, resulting in artifacts and inaccuracies in the voxels.
The attached picture shows what I mean. Note that the left half shows lower resolution to the image edges, and has a weird pixel artifact, despite being captured in the same way from the same source data as the right half.
My core question: Is there any way to force an RVT to generate its highest quality mips?
Ideally, I’d like this to work in a Commandlet as well as the full Editor.
I have tried a few things, with varying degrees of success:
Using IRendererModule::RequestVirtualTextureTiles and IRendererModule::LoadPendingVirtualTextureTiles in a loop, flushing rendering commands and ticking. Seems to slowly work? But is very inconsistent about when higher quality RVT pages come online.
Creating an FSceneView to simulate rendering needs. Very crashy in Commandlet and requires manually controlling a lot of engine operations, which I don’t like.
Using a material to sample out the RVT to a high-resolution target. My graphics team thought this would work, but doesn’t seem to have any impact.
Manually flying the debug camera around the world. Eventually seems to work but, similar to option 1, it’s very inconsistent and uncertain. Plus, not really do-able in a commandlet.
Create Material with RVT Input Parameter, rendering out some property of it (I use the Height of some Heightmaps to Emissive color, but I doubt it matters)
Create a Level with a RVT Volume
In C++, get the RVT from the Volume
Set it to the relevant parameter on the material
Create a render target
Render the material to the target, e.g. UKismetRenderingLibrary::DrawMaterialToRenderTarget
Save the render target to disk
Notice the resolution of the render is inconsistent across runs, and has artifacting
You could look at the existing support for saving streaming mips for runtime virtual textures. This happens in RuntimeVirtualTextures::BuildStreamedMips()
When we do this we iterate over the RVT pages and render each one using the same rendering code as is used by the runtime RVT generation. And then we stitch the pages together to create the final texture asset.
After looking at that code you consider one of the following paths:
Duplicate or modify the existing code to dump the generated texture data into the place where you need it.
Use the existing generation and do an asset export on the generated asset to create .png files.
* build the streaming mip asset through the build button in the RVT component details panel, then
* browse to the streaming mip asset in the content browser and in the context menu do Asset Actions/Export
This will dump the contents to a .png .bmp or .dds file.
You can look at UVirtualTextureBuilderExporterPNG to see how an example of how that works. (UVirtualTextureBuilderExporterPNG is just relaying its internal texture to UTextureExporterPNG).
If you are interested in modifying code beyond that, then RuntimeVirtualTextures::BuildStreamedMips() is generating some raw data in the FinalPixels array.
That data may contain multiple layers (for example BaseColor in layer 0). The offset and format of each layer can be found in FTileRenderResources::GetLayerOffset() and FTileRenderResources::GetLayerFormat().
With this you can extract the data and do whatever you want with it.
Also it might be useful to know that usually in the editor (not cooked build) the best way to get the source data for any given UTexture T is to get T.Source and use one of the FTextureSource member functions on that, such as GetMipImage().
That’s good news. For the commandlet, one reference point is UWorldPartitionRuntimeVirtualTextureBuilder which builds the RVT streaming mips. You could see if it is doing anything in its setup that you are not.
I’ve been trying this approach for a day, and I can’t quite figure it out.
I’ve looked at RuntimeVirtualTextures::BuildStreamedMips(), and it seems promising. I had to modify it to generate something greater than the lowest quality mip, but that wasn’t a major modification.
However, after building, I can’t figure out how to actually get that data into a format I can save to disk. The VirtualTexture->AllocatedVirtualTexture is nulled out by the build, and doesn’t seem to regenerate until some time later (perhaps as prompted by rendering?).
I would have assumed I should use the StreamingTexture property, since we just built that, but its Texture resource doesn’t seem to expose any way to get at what it represents -- the actual data lives in the FPlatformData::VTData, which is all _very_ private. With some additional (much heavier) modification, I am able to expose the VTData, but even then the Chunks of the VT that were created all have no BulkData and no DerivedData, though they do have BulkData metadata with a valid-looking size and whatnot.
At this point, I feel like I’m missing something about how VTs are supposed to work. I can see the SVT in Editor creates a preview of the VT, so _something_ must be able to read the StreamingData, but I just don’t see how or where that occurs.
Any additional guidance would be appreciated. Thanks.
Ah, now I understand what you mean. Yes, the exporter does simplify that part of the operation.
So that helps a lot! I’m able to generate an on-disk asset from the RuntimeVirtualTextureComponent by calling BuildStreamedMips, and then creating a TextureExporter and passing it the StreamedTexture source.
This almost works exactly how I’d like, except even when I’m sampling the RVT onto a 4K texture, I’m still getting much less visual fidelity than I would have hoped. I’ve attached a picture of what I mean. You can see the output texture pixelates much more than the originally attached pictures. I suspect that may just be part of the cost of moving from an RVT to a “real” texture, but I’d love to hear if you have thoughts on how to clean that up (aside from using a bigger texture which, I mean, I will try that).
But, this at least gets my tool into a workable state! Much appreciated!
Yes, I changed the format and am seeing much better results overall.
And, yes, one step of this is the WorldHeight / WaterHeight RVTs Height properties being exported. Essentially I’m making a tool that has “layers” of map data for analyzing and further processing, and those are two of the layers.
It’s all working consistently and with good quality in Editor now. I’m getting an issue where -- not sure exactly but -- the textures appear not to be correctly rendering in Commandlets. But, for now, I’m happy with this outcome and will figure out commandlet if / when it becomes necessary. Thanks!