Poor VT Streaming Results using Niagara SubUVs

Development Asset Creation
, ,
1

Hi,

We have some emitters which use both the Niagara Sprite Renderer SubUV mode and virtual textures. This results in poor and visually objectionable streaming behavior because the VT system isn’t aware of the layout of the flipbook texture. As the animation plays, the UV the material samples jumps around the texture so it kind of outruns the VT system and flashes between low resolution and high resolution pages for the first loop through, which is very visible for FX which play large non-repeating animations like an explosion. Also, the VT cache thrashes when the pool is at capacity since it thinks it can unload parts of the flipbook only to need those pages when the emitter loops back.

We’ve had some success modifying the generated material code to emit additional feedback requests to prefetch other areas of the texture and we wondering if Epic had investigated this issue or had plans to implement something similar.

Our approach is basically:

  1. Add a new SAMPLERTYPE_VirtualPrefetch, which is compatible with any input texture. If the texture is VT, it calls AcquireVTStackIndex but doesn’t insert an actual sample call. Essentially this adds a TextureLoadVirtualPageTable to the shader with no corresponding TextureVirtualSample, for a non-VT it’s a no-op.
  2. If the feature is enabled on the renderer module, we look at Particle.SubUVCoords[0] and Result.Particle.SubUVCoords[1] to figure out which direction the animation is playing and if it’s in fact playing, then prefetch a configurable number of frames ahead/backwards, optionally wrapping around.
  3. Each pixel does one additional VT feedback request based on a dither, this is basically similar to how the VT feedback works using SV_Position to output one texture in the material per VT tile, but another level, where the 3rd feedback sample choses a different future frame per-tile in the prefetch range at the same SubUV within that frame the material samples for the current frame.
  4. This new 3rd dithered UV stored in FMaterialParticleParameters is used for the extra 3rd SAMPLERTYPE_VirtualPrefetch TextureSample inserted in FHLSLMaterialTranslator::ParticleSubUV. Since these UVs are discontinuous across the sprite due to the dither between the list of frames to prefetch, it uses TextureLoadVirtualPageTableGrad with the ddx/ddy of Particle.SubUVCoords[0] to makes sure it asks for the correct mip.

But it seems like there are other ways of doing this though: like for example the VT system could be aware that the texture is used for SubUV and when processing the feedback request on the CPU also request the corresponding pages in the other frames instead of having the shader handle it. However, currently this is difficult to implement since the UTexure2D doesn’t have information about if it’s a flipbook image or what the SubUV tiling is and theoretically the same texture could be used with different SubUV tiling sizes (although likely this wouldn’t be useful to do)

Is this something Epic has looked into? I know the obvious suggestion is “well don’t use a VT here”, but unfortunately (or fotunately) VTs are a large memory savings for us.

Thanks,

Lucas

[Attachment Removed]

2

Steps to Reproduce
Create a simple Niagara Emitter using a Sprite Renderer which plays a looping texture using SubUVs. A single stationary particle is enough to reproduce.

Assign a material which uses a virtual texture for the animated image and run r.VT.Flush on the console to simulate the first load/stream of the texture. Observe that the emitter plays in glitchy and flashing way on the first loop.

[Attachment Removed]

3

Hi,

The short answer is we don’t have a solution for this and don’t currently have a solution in the works.

However your approach seems sound to me, I would lean away from adding a new type EMaterialSamplerType entry, could it possibly be done with a flag on the node instead? That would be less invasive and likely easier to take future integrations. And while on integrations note, just an aside that this would need to be implemented in the New Material Translator.

Then I’m also wondering, is there a way we could perhaps skip the GPU feedback and FVirtualTextureSystem::RequestTiles via the VM / CPU instead? While less accurate, would it be ‘good enough’ and less invasive?

Thanks,

Stu

[Attachment Removed]

4

Hi,

It’s hard to know the correct path (flag vs new enum) without doing the actual implementation, but my gut says that the flag will be less invasive. When Jeremy (owner of the VT system) is back from vacation we can get his take on the path he would go.

Thanks,

Stu

[Attachment Removed]

5

Thanks.

I would lean away from adding a new type EMaterialSamplerType entry, could it possibly be done with a flag on the node instead? That would be less invasive and likely easier to take future integrations. And while on integrations note, just an aside that this would need to be implemented in the New Material Translator.

Thanks for the insights, yeah there is no ideal way. Adding a flag changes the FMaterialCompiler method signatures but then you’re right a EMaterialSamplerType is also difficult in other ways. In terms of the New Material Translator, for sure, but I think we’re waiting internally for that to land since we got kind of faked out by FMaterialHLSLGenerator in the past only for it to not end up being used. :slight_smile:

Then I’m also wondering, is there a way we could perhaps skip the GPU feedback and FVirtualTextureSystem::RequestTiles via the VM / CPU instead? While less accurate, would it be ‘good enough’ and less invasive?

That was my initial thought also and the way I initially approached it, but unfortunately, we have a bunch of materials that also manually do flipbooks directly instead of using the Niagara renderer SubUVs so it’s hard to know. Unfortunately, the flipbook properties of the texture are not part of the UTexture2D itself otherwise it would be pretty trivial to implement this inside the VT subsystem when it processes the feedback.

Thanks,

Lucas

[Attachment Removed]