Lumen GI and Reflections feedback thread

[Miguel1900]

Just a quick first test:

In Epic quality, differences in GI are huge:

UE5.3:

UE5.4 Preview 1:

(Yeah, a little performance drop in 5.4, but in the other hand reflections are slightly better).

When changing booth to Cinematic, they look almost the same, in the right way.

Also noticed that Lumen with r.Lumen.Reflections.Temporal = 0 is much more worse in 5.4. (And yes, disabling temporal accumulation for reflections can be an actual use-case, as it can achieve better results, just combining it with other cvars and parameters to reduce the noise. In fact, if I’m not wrong, we will need to disable it if we want to use DLSS ray reconstruction). First, UE5.3; secondly, UE5.4:

Off-topic: the edge’s artifacts are still there, as nobody knows who is the responsible of this area of development (RT shadows):

[TheKJ]

@Miguel1900

(Yeah, a little performance drop in 5.4, but in the other hand reflections are slightly better)

If you can post a stat gpu shot, interested in seeing where the cost is coming from. Also, kinda looks splotchy on the curtain and we still have lighting leaking.

If I’m not mistaken, traces are done against radiance probes? I feel like this should start including world normal information to prevent this. Ofc, that might only work for bake-able Lumen.

[ZacD]

The couch and curtain look broken. Maybe the cloth shading model isn’t working right?

[Rawalanche]

The difference is indeed big, but in favor of 5.3, not 5.4. 5.4 looks just worse in this comparison. Could you do a path tracer render to see which one is closer to it?

[Miguel1900]

Hi @TheKJ , @ZacD , @Rawalanche , it was a quick test because I’m travelling and won’t have much time to further tests until the end of the month, sorry.

Anyway, I did a quick check: the main difference between 5.3 and 5.4 was due to having a value of 0.8 (or less) in Final gather quality in post process volume, instead of 1 (anyway, the difference shouldn’t be so huge for a delta of 0.2). When 1 or higher, they look very similar, as intended.

5.4, in overall, looks more stable in GI and in reflections too, but a (very) little less performant as tradeoff. Less splotches in general, but they are stronger, I think (like using a bigger sampler).

[TheKJ]

Here was a small test. The project was stress test for all features made in 5.3, then with a bookmark I measured performance. Then copied the project(small in MB, large scene) and switch it to 5.4, same place, same resolution. Not a pretty scene but the performance speaks volumes. Most of the scene is below .4 roughness, very little quad overdraw, high (“60fps”) settings.

Could you do a path tracer render to see which one is closer to it?

did that in this post, but I can go ahead and do an archviz scene soon.

EDIT: Gezz, just realized Lumen GI doubled in cost(both had it in async by default).

[glitchered]

lil performance screenshare. 5.4 preview. stable 48 fps with all the bounces. i didn’t hit that in the daily with the floor reflection enabled. lights spammed. shadows softened. no dlss ofc. engine firing on all cylinders. this is great.

edit: nvm. i shot at 720p internal. my stupid blueprint, again. xP either way… the good features are in. and it’s absolutely usable to make good looking games above 30 fps (i can smooth sync 36 to my panel).

[Matthew.Ivey]

Hey!

I saw your post in the 5.4 Preview thread and wanted to let you know that we’ve already logged this issue internally. This actually stems from Ray Traced Shadows - From a previous fix on the shadows side, we get better shadows but also reveals this artifacting. For tracking of the issue, please see UE-200307 once available.

[Krzysztof.N]

@TheRealBiomes

Unfortunately doing this severely underoccludes the asset so we are left with a very flat looking foliage asset in our test cases, opposed to an overoccluded one. I strongly suggest considering adding a switch for this or generally looking at the described scenario; if most of the assets geometry has an opaque material assigned to it (what % I’m not sure of), any parts with a two sided foliage material will be overoccluded. Lumen weakens the distance fields when the ratio is within a certain range, but this doesn’t take into account not all meshes will fit in that range.

I see, so to make sure, you are looking here for a three state switch: [auto, two-sided, one-sided] instead of a current checkbox “Generate as two-sided”?

@TheKJ

A Cvar that would really help things out would to limit reconstruction from Bilateral and Reconstruction samples from affecting lower roughness like 0.0, 0.01.

That’s an interesting point. We have something similar in reflections, where reconstruction and bilateral filters are scaled by roughness, and temporal accumulation is reduced for mirror like surfaces. I’ve made a note to look into it to see if it can be improved.

@TheKJ

Lumen Reflections do not jitter in a way where 0.0 reflections can be constructed to the main internal resolution as frames pass on low roughness, if they were, we could get high res reflections in around 5 frames. I guess this was unfortunately handed off too.

We already have jitter from screen pixels and not sure what else could be jittered for roughness=0.

[TheKJ]

We already have jitter from screen pixels and not sure what else could be jittered for roughness=0.

Sorry, there was a typo. I meant to say this is handed off to TSR/TAA. Which is problematic for accessibility(Temporal upscaling and Motion Blur included). Temporal effects such as frostbites SSR and warframe SSAO are low resoltion but have their own systems in place to upscale and accumulate. Advil recently advised against Temporal AA and Upscalers and Blur Busters also trying to bring awareness to devs about this issue. Per effect ghosting isn’t a issue, becuase it doesn’t cause the entire scene or even effect to blur.

DLSS RR uses a specular motion vector buffer based on specular hit distance from the first bounce. Maybe a combination of short term interpolation as a fallback. Also, I would like more control as to how fast past reflections are faded away. I have done some slight jittering with GXXsampling but It didn’t end up working becuase past frame are discarded/faded too fast.

Maybe a value that controls how fast/slow past channel information cuts off over time too?

Now, again I really appreciate the work done on performance regarding reflections but I do have a concern over the major jump in performance cost in the ScreenReconstruction method which jumped significantly. Also, I find screen traces to lack performance/quality seen in other engines and implementations as Frostbites SSSR and this AMD hybrid solution demo(which might be optimized for consoles due to RDNA2 focus).

I have carefully measured performance on each scene, these aren’t meant to put Lumen down, but rather make a point about cost/quality. In in all my test, AA is off, clipmap extent is at 0.0, r.UpscaleQuality is at 0 and assume everything is on default high settings unless stated otherwise.

Frostbites documented SSR uses TAA, which was kinda needed at the time(PS4), but modders and recent games have taken that solution to remain to stable without TAA and at very little cost, we are talking maybe 12% of a 3060 at 1440p in a very wet environment, no world normal limited in trace directions, little to no smearing, both High and 0.0 roughness remain very stable without TAA and no major dithering is produced for that matter. Even in fast traveling motion, no smearing is as pronounce compared to Lumen Screen Traces.

Test scene

Path Traced:

I’ll go ahead an omit blurry 0.0 reflections from the comments below since this seems like it will be tackled soon so I’m not worried about that.

• 1440p, 3060, cost 4.3ms with the setting I stated. Looks fine if not pretty close to path traced but looks more smeary in comparison to frostbites SSSR in motion. That smear issue is still present in every one of these scenarios, which could be fixed with less frames or better use of specular motion vectors.

• 1440p, 3060, cost 1.5ms with r.Lumen.Reflections.ScreenSpaceReconstruction 0 and r.Lumen.Reflections.BilateralFilter 0. Anything not moving with roughness above .02 displays very noisy and visually sporadic reflections. The reflection output is rough on the eyes because we can easily distinguish 4 real pixels represent a output pixel. Ugly(sorry😬) dithering patterns also become visible on cutoff edges.

• 1440p, 3060, cost 3.5ms with r.Lumen.Reflections.ScreenSpaceReconstruction 0 r.Lumen.Reflections.BilateralFilter 0 and r.Lumen.Reflections.DownsampleFactor 1 The noise mentioned in bullet above is now 80-90% cleaned up. Dithering patterns become less visible but are still present. The noise can be further softened with r.Lumen.Reflections.Temporal.MaxRayDirections 32 at no cost.

• 1440p(r.ScreenPercentage 40, 1027x579), 3060, cost 1.04ms with r.Lumen.Reflections.DownsampleFactor 1 r.Lumen.Reflections.BilateralFilter.StrongBlurVarianceThreshold 0.1
  Everything in the scene looks low res but reflections are clean with no noise, proportionally blurred and clear on differentiating roughness, even 0.0 looks clear my display which is 1440p.
  A slight problem arises, the reflections above 0.07 look too blocky which could be taken care of by a spatial upscaler(rather than nearest) Like r.Upscale.Quality 5 or FSR1+Edge correction seen in the AMD hybrid SSSR+RT project(It would need a Cavr to control what roughness receives this spatial upscale).

Taking advantage of the last scenarios performance would entail computing downsampled versions of the g-buffers and spatial algos, but I can’t imagine that being more expensive than all the other scenarios I listed. Also r.Lumen.Reflections.RadianceCache 1 feels misleading as in my experience, it actually increases the cost of Lumen Reflections (lowest .25ms, highest .50ms).
I feel like this would then be better paired with the power of offscreen traces.

[glitchered]

you got this test scene as a download? i’d throw my eyeballs at it and judge. i mean… i would not render ssr anyway. i’m over that one. it raytraced or bust. i don’t like ssr artefacts.

what you mean by specular motion? that’s hard to get. there’s no first bounce. specular is a analytical light term created at hit point. if you look at it, it’s computed directly. if you look at specular highlights in a mirror you have a reflection bounce and then the specular is computed for the surface in the mirror. the hit point. getting correct motion for that is a like a buffer for every bounce and computing reflection vectors for every motion sample. expensive. and it can differ based on the roughness and the random reflection vector. roughness creates a random reflection vector/ray.

your pathtraced screenshot is just a reflection. and ofc the rougher the surface is the more scattered the rays wil be. in the game engine you get one ray per pixel per frame and temporal stabilisation via motion history. it’s still randomly sampling roughness. it is what it is. the bilateral is denoising and blurring it. the downsampler changes the resolution. and screenspace reconstruction is a goofball i don’t even know. i don’t use screenspace reflections. it will never reconstruct what it can’t see and expose hidden surface artefacts. facts.

so you may have to accept that you have scattered rays that compute temporal noise on very clean surfaces. in a game world with detailed texture you may not see as much.

[TheKJ]

what you mean by specular motion? that’s hard to get. there’s no first bounce.

First off, Nvidia spoke about this.. You say it’s expensive but we are already dealing with that, aren’t we? The goal is a alternative.

i mean… i would not render ssr anyway. i’m over that one. it raytraced or bust. i don’t like ssr artefacts.

If done well, you shouldn’t get any and you get free performance that’s usually pretty contextual in terms of when screen information isn’t on screen: it isn’t on screen(basepass etc goes down as raytracing kicks in higher ms). I would check out AMD hybrid SSSR+DXR demo. Because your just throwing away free performance instead of “working smarter”.

your pathtraced screenshot is just a reflection

It’s just reference as to how roughness should scatter across roughness fresnel. Which certain Lumen reflections Cvars can skew how closely Lumen replicates it. I know how sampling works in engines, the problem is that other engines do it faster and in more visually pleasing ways. Regarding the test I did, this was the scene that gave some context. Which is important for any raining scene.

temporal stabilisation via motion history–
so you may have to accept that you have scattered rays that compute temporal noise on very clean surfaces.

It’s not a matter of accepting, it’s already possible.

you got this test scene as a download? i’d throw my eyeballs at it and judge

Aside from personally made assets, isn’t everything a download? I made the scene recently out of some Nvidia projects. Not sure what this meant from your view?

[glitchered]

@TheKJ

If done well, you shouldn’t get any and you get free performance that’s usually pretty contextual in terms of when screen information isn’t on screen: it isn’t on screen(basepass etc goes down as raytracing kicks in higher ms). I would check out AMD hybrid SSSR+DXR demo. Because your just throwing away free performance instead of “working smarter”.

well… i know it doesn’t work or is it smart. even hybrid doen’t change much. look… screeen space portion.

argueable i could use hybrid to safe on the visible facades. but… if the floor was a puddle i’d have to hybrid raytrace 75 percent of the screen anyway and blend it at the ss edge. and don’t get me started on the characters. the complexity is off the charts for hidden surface artefacts. hybrid, peek thru holes, check if you see screenspace pixels, motion tracking and all the fakery sh*t doesn’t need to be done if you have a bruteforce method/raytracing and just do it.

[CROSE48]

Enabling Substrate in the 5.4 preview appears to mess with the GI. Any help on this?

[jblackwell]

Sebastian Hillaire in the substrate channel spoke to this a bit: part of it is simply a product of the fact that substrate and the legacy materials aren’t actually guaranteed to be 1-1 matches for each other, and so you may have some lost/gained energy. Not to mention, the system is still in experimental, so I wouldn’t be surprised if you were seeing strange behaviors with radiance.

I would check your lumen scene view first however, to see if the card capture is registering significantly different materials.

[CROSE48]

Yeah I checked that and they look the same. I’ve just made some screenshots. I’m also discusing this with Seb in the Substrate feedback thread.
So I take them using the “High resolution screenshot” tool in the Viewport but as you can see from the manual screen grabs there is also a strange blurring happening when Game Mode is On.

UPDATE: I can confirm that enabling “Allow Static Lighting” does fix the problem

[TheKJ]

I decided to dig a little deeper into performance comparisons, maybe @Krzysztof.N or @Daniel_Wright can share some insight as to how Lumen reflections work. I decided to use a hardware/API inspector to get the exact timings on FB SSSR since it is rather visually appealing and pretty performant for real-time.

Test scene specifications:

Frostbite modded(higher res) SSSR channel, 1.4ms@1440p-Desktop3060
Most of the reflections visible in this channel are overlapped(covered) by the next processes.

I tried to match it with the scene above, this was “2.2ms”@1440p on the same 3060.
Settings: High preset, bilateral and reconstruction off(0), downsample factor 1, clipmap extent 0.0, r.Lumen.Reflections.Temporal.MaxRayDirections 32.

The reason why 2.2ms is in quotations is becuase I’m just trying to measure the screen tracing speed. In the same scene, when r.Lumen.Reflections.ScreenTraces 0 is inputted to the console, Lumen reflection cost drops to 1.5ms. So the actual screen traces are costing around (2.2-1.5=)0.7ms.
That’s a lot faster than r.SSR.Quality 3 which is the only ScreenSpace solution that offers elongation but cost 1.7ms+more temporal instability(without TAA etc) due to jitteriness and noise:

Conclusion after data: My point is that SSR in unreal needs to be replaced with Lumen Reflection Screen Traces solution with the settings I applied(downsample factor etc) as it’s much more efficient. It would also serve us well is to allow us to have this screen trace solution at this level of quality&cost without needed to bump up actual SDF/Triangle traces. Also, this would save us the cost of reconstruction from running on screen traces(since at that quality, it really isn’t needed).

Now, the next step is profiling non-screen traces. We aren’t always lucky to have on-screen information so I’ll give my thoughts on the same scenario with no screen traces.

For the purposes of keeping this informational post short, I’m only going to give timing results on visually stable settings with no TAA/DLSS etc.

Non-Screen Trace timings

Default High settings-Software tracing-Desktop 3060:

• 1440p 4.80ms(3.4ms without SSreconstruction(-30%))

• 1440p(r.ScreenPercentage 50) 1.5ms(1.2 without SSreconstruction(-20%))

Default High settings-Hardware tracing-Desktop 3060:

• 1440p 3.02ms(1.5ms without SSreconstruction(-50%))

• 1440p(r.ScreenPercentage 50) .70ms( .50ms without SSreconstruction(-28%))

I’m wondering if hardware acceleration always boosted reflections like this or is this the new HWRT optimizations for 5.4? Either way, these test are showing issues with SSreconstruction and showing the value of upscaling low resolution traces after SSreconstruction runs at a lower resolution:

Trace>>SSreconstruction(ofc with the exception of low roughness which needs separate jitter if TAA etc is OFF)>>SpatialUpscale reflection channel.

It’s a small test in a small scene but it’s highlighting some issues that could blow up in bigger scenarios.

Side note: Enabling restir on Lumen results in no GI but increased cost and the GI is visible in the reflections. Not sure what’s up with that. I’ve done a lot of trouble shooting but it just doesn’t work.

EDIT 2:

Quick note, 0.7ms isn’t the full story. That only disables the screen tracing, r.Lumen.Reflections.Temporal 1 in that scene cost .22ms and would still be needed. So in reality it’s 0.7ms+anything in the 1.5ms that is needed(for instance, .30ms of that 1.5ms is tracing voxels, etc, etc)

EDIT3: UE5 is inconsistent with temporal stochastic SSR, which until recently it randomly worked after a couple years of it not. I’m pretty sure this makes SSR faster in unreal.

[Arkiras]

@Krzysztof.N Please consider making matching RaytracingGroupIDs change for each instance of Packed Level Actors. Or at least expose the setting to blueprint for ISM components. Currently, whatever IDs you assign to the meshes are used for every instance of the PLA, leading to this annoying behavior where cards are merged into a big box around all the PLAs instead of how they were assigned inside the PLA:

I thought I could fix this myself by assigning the GroupID manually in the construction script, but this can’t be done from Blueprint because there is no function available for setting the raytracing group ID for individual components, it can only be done at the actor level:

[KWMHP]

Hi,
Is the “High Quality Translucency Reflection” setting supposed to work without Lumen GI? I’m only using Lumen reflections but this setting produces extreme artifact (actual colorful rendering glitches) on all transpareny surfaces without Lumen GI.
I hoped this would help to improve transparency quality with less performance penalty than RT translucency.
From a UI perspective it’s a bit counter intuitive to have it in the reflections tab, if it’s actually linked to GI.

Issue is fixed in 5.4

[Miguel1900]

Are you in 5.3, right?
In 5.4 I tested it a while ago and it seemed to be already correctly fixed and working