Supporting DLSS Ray Reconstruction with Lumen Reflections' DownsampleFactor > 1

Development Rendering
, ,
1

Hello!

Quite aware that Epic is only part of the puzzle here and likely this falls on the plugin (whether the UE plugin or the base SDK), but any help would be fantastic as we’ve not had much of a response from Nvidia.

The issue is that when we enable DLSS-RR while ``r.Lumen.Reflections.DownsampleFactor``is set to anything but 1, we experience a very large amount of speckle noise on any material that attempts to reflect any light. I suspect this is because the tagged textures used are of a size unexpected by the ray reconstruction process, but I’m not knowledgeable enough to know which texture and how to compensate for the difference, there’s only really options out of the box for motion vector scaling but that doesn’t seem applicable here. We’ve isolated that only affecting the CVar above causes issues.

I imagine there might be something around tagging a pre-scaled Specular Motion Vector Reflections resource if one’s available/can be generated? Is that even relevant here? Apologies rather grasping at straws somewhat.

Any help would be fantastic, hoping you’ve had another studio with the same issue or something along those lines.

Thank you,

Billy Fletcher

d3t Ltd

[Attachment Removed]

2

Steps to Reproduce

  1. Integrate latest Nvidia DLSS plugin
  2. Turn on Lumen Reflections in the project
  3. Enable DLSS and set DLSS-RR mode to On.
  4. Run the game (editor fine)
  5. Toggle the Cvar `r.Lumen.Reflections.DownsampleFactor between values of 1 and 2`
  6. Notice clear noise effects on lit, reflective surfaces
    [Attachment Removed]
3

Hello, could you send over a Pix or a RenderDoc capture of a frame showing the noise, along with a screenshot? That would help in getting a closer look at where the noise is being introduced.

[Attachment Removed]

4

I’m somewhat trying to avoid using a confidential question and captures/screenshots would require this, but upon trying to generate a repro project, it appears that the cause of the issue is related to engine changes on our end, possibly the cherry-picking of this commit: https://github.com/EpicGames/UnrealEngine/commit/fead0b4f3cfc6262371837867d16e8aea3345481 into 5.7.3.

Will update this thread when I can confirm it’s the cause. I cannot get the same effect to appear in 5.7.3 vanilla but it’s possible I’m missing some specific combination of CVars. Please don’t close for now and will update to ask relevant questions when I can isolate the cause.

[Attachment Removed]

5

Hello! Apologies for the delay! So after some more investigation, it appears unrelated to any engine changes on our end and can be reproduced in a vanilla 5.7.4-release version engine.

It appears that the issue is related to DLSS-RR’s recommended enable function, “UDLSSLibrary::EnableDLSSRR()” disables other forms of denoising and temporal reflections, as far as I understand it, this makes sense as DLSS-RR is meant to replace all other denoising passes with a single pass, its own. However it appears that either ray reconstruction isn’t running at all or has some negative interaction with the “DownsampleFactor” and possibly expecting different buffer sizes or something like that. I’m definitely not a proper rendering engineer so I’m not sure if this is a bug or expected behaviour, nor am I sure of the impact of some of the options that ‘fix’ it (see step #9) or whether they’ll invalidate DLSS-RR entirely.

Also for context, we’re using DownsampleFactor for a scalability setting as it provides a good way of having lumen reflections while also being more performant (for a 'High’setting) and we were struggling to find lumen reflections related configurable values that provided meaningful performance/quality tradeoffs.

Repro:

1. Create a new third person project using 5.7.4-release

2. Add DLSS v8.5.0 (or above) plugins to Engine/Plugins/Marketplace/DLSS

3. Rebuild engine

4. Enable all DLSS plugins except Movie Render Queue

5. Change material instances in scene to have high metallic/reflective properties

6. Run PIE

7. Set cvar: “r.NGX.DLSS.DenoiserMode 1”

8. Set the following cvars in line with the behaviour in “UDLSSLibrary::EnableDLSSRR()”: “r.Shadow.Denoiser 0”, “r.Lumen.Reflections.ScreenSpaceReconstruction 0”, “r.Lumen.Reflections.Temporal 0”, and “r.Lumen.Reflections.BilateralFilter 0”.

9. Set “r.Lumen.Reflections.DownsampleFactor 2” and observe large amount of noise. Note that toggling “r.Lumen.Reflections.DownsampleFactor” to “1”, “r.Lumen.Reflections Temporal” to “1” or “r.NGX.DLSS.DenoiserMode” to “0” will reduce the noise dramatically.

I’ve also tried to provide 2 captures, one with “r.Lumen.Reflections.DownsampleFactor 2” set (w/ the issue visible) and one with “r.Lumen.Reflections.DownsampleFactor 1” set, for comparison.

[Image Removed]

[Attachment Removed]

6

Hi Billy, thanks for putting together this detailed write-up. I took a look at the captures, and it seems that DLSS-RR cannot deal with the way we handle downsampled reflections. When you set r.Lumen.Reflections.DownsampleFactor=2, we trace reflection rays at half-resolution before upscaling them again in our own reconstruction pass, but that information is not passed on to DLSS-RR. Since we are not the official owners of the DLSS plugins and your issue appears to lie within the DLSS codebase, I recommend reaching out to Nvidia, as they should have more insight into your problem and how to resolve it. If you have any further questions, please let me know, and I will do my best to answer them.

[Attachment Removed]

7

Thanks Tim, will let you know where we end up whenever Nvidia get back to us then, appreciate the insight/will pass it on.

[Attachment Removed]

8

Ok, that sounds good to me. I will close out the ticket for now, but please feel free to file another one once you hear back from Nvidia.

[Attachment Removed]