Questions about RVT and SVT built from RVT volume

Development Rendering
, ,
1

Hey Epic, we are considering the feasibility of using the SVT built in RVT volume for distant objects. For example, we make the HLOD landscape meshes do not generate their own textures but directly sample the built SVT, so that we can save some disk storage. We would like to have thorough understanding of how it works, so we can control it better. We have questions about it:

  1. We can set the build levels in the RVT volume settings. If we set it to 4 and build SVT, does it mean the highest 4 MIPs use the built SVT, while the other lower MIPs use RVT?
  2. If the factor which decides RVT falls back to SVT is MIP level, can you point me to which function is responsible for this? Is this info embedded VT feedback buffer for VT page requests? We may need to do some trick here to force which MIP to use.
  3. I noticed that we can use MIP bias to force the material to sample higher MIP. For example, if the landscape uses MIP 1, whereas the MIP bias is set to 2 for the grass (which samples RVT) on the landscape, which means grass samples from MIP 3. In this case, do we end up having 2 MIP for the same location? If so, does this consume more memory in the physical memory pool?
  4. We noticed that the built SVT is in a Virtual Texture Builder, and we cannot directly access the built SVT in materials. Is there a way to export the built SVT, so that we can directly use the SVT in materials?

BTW, we are using UE 5.4.4.

2

On top of my question 3, I also would like to know the MIPs of RVT tiles. Think about this, the original RVT is big which has a lot of MIPs, let’s say 0 ~ 10. And we bake the SVT for the highest 4 MIPs. Then theoretically, when using the RVT in the game world, MIP 0 ~ 6 will render the RVT tiles, while MIP 7 ~ 10 will load the SVT tiles to memory. Hope I understand this correctly.

If a close landscape uses MIP 0, the system will render MIP 0 and store it in memory. In the meantime, does it generate MIP 1 ~ 6 by down-sampling rendered MIP 0? Or it does not generate MIP 1 ~ 6 at all?

3

I’ll attempt to answer your questions, and then I have a suggested approach for use of SVT for distant objects.

  1. Yes the build level setting determines the number of mips in the SVT, and the SVT is used to provide the top (course resolution) mips of the RVT
  2. We create a FVirtualTextureLevelRedirector which redirects page requests to either the SVT or the RVT
  3. Yes if you sample different mips at the same location, the virtual texture will populate both and this will take the corresponding space in your physical pool. In reality this usually isn’t a bad thing, and is better than sampling at the incorrect mip level for either landscape or grass.
  4. The context menu for an SVT asset in the content browser has an asset-action/export option, which exports to an image file, which can then be imported as a texture asset.
  5. Each runtime generated page is generated by rendering directly and not by downsampling another mip level. The SVT pages are generated (offline) by rendering the full volume to create the finest mip level pages, and then downsampling to create the other mip pages.

For the rendering of distant objects, one approach is to create two RVTs covering the same space, one (A) for the usual usage, and another (B) for distant objects (HLOD etc).

A is created as normal, and an SVT is set up.

B is created to cover the same volume as A, and has the same tile/border sizes as A, but has a final resolution the same size as the SVT of A

B is set to point at the SVT of A, but doesn’t have the build level settings for creating its own SVT

B never renders pages in cooked builds because it always loads the pages from the SVT.

Normal landscape/grass etc samples from RVT A

HLOD/distant objects sample from RVT B

A reason why this is a good idea is that RVT B will always give valid distant pages, without the need to render them at runtime (which would require distant landscape to be loaded).

I think this approach works in UE5.4. But some features were added to 5.5 and the upcoming 5.6 to make it even easier to use.

Best regards,

Jeremy

4

Thanks for your answer and new approach, they are helpful, we are discussing about the approach you suggested.

Can I ask a bit more about detail?

Regarding answer 3, if sampling different mips at the same location results in both mips stored in physical pool, it indicates more memory consumption. If the landscape and the grass at the same location use the same mip by default, adding mip bias to the RVT sampler in grass material requires more mips, which should increase memory usage. However, I saw in our project, after I added mip bias to the RVT Sampler in grass material, the memory usage went down (I observed it in r.VT.Residency.Show). Do you have hypothesis what happened here?

Regarding answer 5, are there anyway to generate high level mips by downsampling instead of always render/load the exact required mip level? There is another issue in our game when driving fast, the speed of RVT update is not fast enough. Sometimes I can see we are always running on blurry road, and after stopping, the landscape around the car was updated gradually. We tried to increase r.VT.MaxUploadsPerFrame and r.VT.MaxTilesProducedPerFrame, this helps a bit, but we cannot abuse them as too high r.VT.MaxUploadsPerFrame introduces too much cost to performance. Do you have good solution for this type of issue?

5

> If the landscape and the grass at the same location use the same mip by default…

By default landscape and grass probably do not use the same mip. Grass will likely have more verticality, and since the UV space is a top down projection, the increased gradient will likely lead to grass requesting a finer mip than landscape. (Landscape would also request a finer mip at cliffs). So applying a mip bias to grass is quite likely to reduce residency.

> Are there anyway to generate high level mips by downsampling instead of always render/load the exact required mip level

There is no supported path for this, and I think there would be a couple of issues. First it would only work when we have all 4 “child” pages for the page that we want to generate from downsampling. But that might be OK in some scenarios. However, more problematic is that the course mip that we are generating has border pixels that will be outside of the area covered by the child pages.

> Do you have good solution for this type of issue?

I think it might be difficult to solve this. It sounds like there is a strict bandwidth of page generation that you need to keep to (for performance), but that you need more bandwidth than that to keep to you visual goal. It’s possible that RVT might not be a good fit in this scenario.

Any RVT based solution probably relies on being smarter about the pages being updated. Maybe you could increase the resolution covered by your baked streaming mips? Also ensure that you are separating out the budget for runtime and streamed page updates using “r.VT.MaxUploadsPerFrame.Streaming”.

Another thing to try might be to rely on more CPU driven preloading of pages in the direction where the car is going. UE5.6 will have a new RequestPreload() function on the runtime virtual texture component that can help with that. But this is only a thought, and not something I have seen done, and it cannot magically change the perf/visual bandwidth trade off.

Best regards,

Jeremy

6

Thanks for your patience in answering. We learned a lot from you. At this moment, we are working on RVT (technically, the whole VT system) performance issue, we may come up with new questions to ask soon, so it is a bit too early to close this question now, we would like to leave this open for a while. Again, thanks for your support :heart:

7

I’ll mark the issue as pending. Sometimes there is an auto close process, but feel free to reopen if that happens.

8

Hey Jeremy and Epic, thank you for this solution. We implemented it, it’s simple and elegant!

It seems to have a side-effect, though, at least with 5.4.4: It breaks an unrelated RVT :frowning:

The system seems sensitive to the number of RVTs or RVT volumes or sharing of SVTs.

Our main landscape HLOD RVT + SVT situation is solved with the suggested approach, i.e. creating another volume (same size) with a new RVT, but with the shared SVT and the same number of mips as the shared SVT has. Perfect.

Elsewhere in the game, however, we have other RVT volumes, with other RVTs with their own SVTs. One of them breaks the moment with plug in our new shared RVT into the unrelated HLOD material.

Do your engineers perhaps have a recollection of a CL in 5.5+ that might be addressing this and that we might not have?

Given our time pressure, we might not be able to repro this in vanilla 5.4 in time. We can try, though, if this doesn’t ring a bell.

Thanks for additional thoughts, Pavel

9

Hi Pavel,

I don’t immediately recognize the issue that you describe.

Is it possible that the new HLOD RVT takes your RVT count to greater than 7? That was a restriction that we only lifted in the upcoming UE5.6 release. I would expect that bug to appear as soon as the 8th RVT volume is placed, but you mention that the issue happens when the HLOD material is updated.

What are the symptoms when the bug happens? Is it always the same RVT that is affected? What are the formats of your RVTs? If it happens in editor, then do any of the RVT thumbnails (which live update with their current state) look broken?

Best regards,

Jeremy

10

Hi Pavel,

The changelist that changed the 7 RVT restriction was: 40429664

There are a couple of relevant follow up and bug fixes that I’m aware of: 40432542, 41691234.

Another way to increase the restriction from 7 to 15 without such a large code change would be to modify FPrimitiveVirtualTextureFlags so that RuntimeVirtualTexture_BitCount = 15 and both bRenderToVirtualTexture and RuntimeVirtualTextureMask are uint16. It looks like FScene::RuntimeVirtualTexturePrimitiveHideMaskEditor and FScene::RuntimeVirtualTexturePrimitiveHideMaskGame and their accessors would need to be updated for uint16 as well.

A simple test to determine if this is the issue before making bigger changes would be to change the checkSlow() in FRuntimeVirtualTextureFinalizer::InitProducer() to a check().

All the best,

Jeremy

11

Hey Jeremy, thank you for a fast answer.

As a matter of fact, we do have 8 volumes + RVTs now, but 2 of them should not be loaded when it’s broken. We’re working on confirming that.

Let’s list all our RVTs anyway:

(names edited)

For main landscape:

* Volume1: RVT_Color / SVT_Color

* Volume2: RVT_Color_Distant / SVT_Color (!!!)

* Volume3: RVT_Displacement / SVT_Displacement

* Volume4: RVT_Height / SVT_Height

For location 1:

* Volume5: RVT_Loc1_Color / SVT_Loc1_Color

* Volume6: RVT_Loc1_Displacement / SVT_Loc1_Displacement

For location 2:

* Volume7: RVT_Loc2_Color / SVT_Loc2_Color

* Volume8: RVT_Loc2_Displacement / SVT_Loc2_Displacement

Our latest breaking addition is Volume2 + RVT_Color_Distant, sharing SVT_Color, and it is used only on the HLOD material.

RVT_Loc1_Color is the broken one. It seems that none of the actors (projectors) don’t render into it, and when sampled in Location1 on the floor, it returns black.

Strangely enough, RVT_Loc1_Displacement and all other RVTs are fine, as are all the SVTs, which we didn’t re-build in a long time. Also pages on RVT_Loc1_Color falling into its SVT are fine, only the near ones are black.

In editor, I didn’t have to remove any volumes, just change the HLOD material from using RVT_Color_Distant back to RVT_Color, and this fixes RVT_Loc1_Color.

Do you have a CL of where you lifted the restriction?

In the meantime, we’ll remove 1 volume + 1 RVT and try again.

Have a nice evening,

Pavel

12

Most obliged! I’m trying the minimal change, then I’ll look at Perforce.

Worth noting that I’ve been running under debugger the whole time and I’ve never observed an assert…

13

Hey Jeremy. Upping RuntimeVirtualTexture_BitCount to 15 did this for us, we can work with this.

I will not go into merging the CLs, they have some dependencies.

Thank you very much, we’re done with this problem.