TSR feedback thread

General Feedback & Requests
,
42

As a sanity check, I had a friend send me one of his grooms, and then popped said asset into a completely empty project. I can send it over if you would like to look at it too, it basically has three scene captures pointed on an actor that’s moving via animations.

They have some slightly different settings than mine, but we both see the smearing (he’s on installed, I’m on source) pretty heavily. We’re kind of puzzled as to how this is an issue in such a base case.

I’ve attached a GIF of what we’re seeing. This is from the output of one of the render targets. The movement in viewport looks fine.

UnrealEditor_gPamXGehn2

43

Hello!

Sorry for the delay! I suspect motion vector problem: just do a DumpGPU and have a look at how the motion vector looks in the TSR DilateVelocity pass used in your scene capture. This trick on the scene capture will allow you to name each individual scene capture you have to be clearly identifiable in the DumpGPU: TSR feedback thread - #33 by Guillaume.Abadie

44

That smearing definitely looks a lot like the smearing issues I’ve been dealing with in a project I’m working on. I made a thread and have also found that this comes back to motion vectors, except we run into issues because we’re combining TSR/TAA with transforming instanced static meshes that are animated using WPO animation, where it seems fixing one issue breaks the other :sweat_smile:

45

Hi @Guillaume.Abadie

I ran into a special problem. Our scene setup is using the Ultra Dynamic Sky plugin. For reducing flickering in our scene, I used the cvar r.TSR.ShadingRejection.Flickering.Period=10 to reduce flickering. Unfortunately when changing the light time, you can visibly see that there are “light artefacts”. And the scene needs more time to settle until it is gone.
Do you have any idea what is causing this? I will try to see if I can reproduce it with an Unreal light source as UDS is quite a special thing.

Edit: This is caused by using a noise node for creating a normal map. Using a static noise solves this. The effect is very obvious in the shadow and if you apply this noise directly you can see that it is only noisy and living in the shadows.

Edit2: issue was actually mipmap gen settings of the normal texture set to “NoMipMap”. Choose any other value here to get rid of the error.

1 Like
46

hi, @Guillaume.Abadie

I found that if I set the “Enable alpha channel support in post processing (experimental)” option in the render settings to “Allow through tonemapper”, there will be black ghosting in the SceneCapture viewport, Game viewport is OK.

企业微信截图_16921785104820
企业微信截图_169217851048201092×218 15.7 KB

47

@Guillaume.Abadie

This is a video link with ghosting.
The character is transparent and the gray background is superimposed by me in the video.

48

Did you select output velocity on the transparent material?

49

The character is not made of transparent material, there is only one character in the level, and the background is transparent. I used Scene Capture (NDI Plugin) to output this video with a transparent background.

50

Hi Songfei,

Thanks for bringing this to my attention! Would you mind sharing a DumpGPU with r.DumpGPU.Root *TSR* when the artifact happen? GPUDump Viewer Tool in Unreal Engine | Unreal Engine 5.0 Documentation

51

@Guillaume.Abadie
Dump file here

1 Like
52

I fear your dump hasn’t captured the problem, since it seams to have a completly empty scene with completly black pixels.

Dump:
	TestTSR - Windows - D3D12 - ++UE5+Release-5.2-CL-26001984 - 2023.08.18-17.05.05
	#display_input_resource(188,'00000000e9620260.mip0');
Pass:
	TSR UpdateHistory(Quality=Epic) 5894x3762
Input resource:
	SceneColor

image
image1920×1080 42.5 KB

54

hi, @Guillaume.Abadie

I was very depressed when I heard this. I tried it several times, and the result was the same.

I made a very small UE5 project that recreates this problem.

This problem does not appear in game viewports, only in Scene Capture viewports,

I make UMG Widget to display Scene Capture Image.

step:

only run game

UE 5.2.1 project file, only 4M :

1 Like
55

hi, @Guillaume.Abadie

reproduce the problem?

56

This is also true in our issue too. The Scene Captures will always smudge heavily.

We’ll look into this as well, and post an update.

I’ve attached our google drive with the test example. Steps are easy to recreate:

  1. Put FurTest folder into a Content folder for a project.
  2. Open the Main map in the FurTest folder and run the simulation.
  3. Open the Capture1, 2 and 3 Render Targets (in the Captures folder) and observe smearing as the character moves.
1 Like
57

It seems that this fix has introduced a TSR flicker rejection bug with screen traces.

When outputting to CustomSSR, TSR does a great job at removing a lot of the noise from screen-traced emissives. But when outputting to TemporalAAHistory, the noise problem is exasperated. This is obviously most noticeable with small screen-sized emissives that are solely lighting the scene, but it’s also noticeable for any form of screen-traced emissive.

See the attached videos showcasing this side effect of the ScreenSpaceRayTracingInput change. There is also a chance I’m missing something trivial, maybe a missed setting? Did this fix change the way TSR flicker rejection works?

Using the New TemporalAAHistory Output:

Using the CustomSSR Output:

(I also commented on the Git commit before I was pointed to this thread. If needed, I can delete the GitHub comment for the sake of keeping everything in one place)

58

Unreal needs user controllable per-pixel velocity output from base pass. Always needed that, but now with TSR it is even more so. Not only it would address some pronounced issues but would open up way for range of effects.

1 Like
59

@Guillaume.Abadie I was reading up on this paper, I think you should take a look at it:

This paper includes how they achieve a TAA algorithm that looks so much better than anything I’ve seen in the past 5 years including the current TAA in Unreal and DLSS. I’m talking lightyears ahead.

“That is, we use the raw render of the previous frame and the current frame, but nothing more. That way, we prevent any long ghosting trails caused failed rejects, and we reach a stable and final result in only two frames, making our technique very responsive.”

This right here, uses a very cheap TAA while preventing ghosting. The latter was something that TSR aimed to achieve but it still has issues with foliage even if the material is outputting velocities. And TSR is not cheap.

Unreal Engines’ TAA default is 8 jitter samples and infinite frame re-use. Same goes for upscalers like DLSS. There is no combination of CVARS to fix unreals TAA. UE’s TAA is simply outdated and terrible in motion, so basically all the time except in screenshot :unamused: . (unless it’s on at 4k as we discussed together already).

Make no mistake, the studio that owns Decima updated their latest game with a worse version of their TAA ruining motion clarity after some idiotic complaints about slight shimmering on grass which should have been mitigated with mips, and shaders.

This is miles ahead of anything I’ve seen, I’ll most likely be trying to work on implementing this in unreal if I even can as I am not an Engine Programmer.

I do not care about slight shimmering. Not everybody should not be subjected to motion smearing in their game which has been the case for many years in deferred rendered games in UE.

I want good motion clarity while handling specular issues. That is something Unreal has not offered at 1080p with deferred rendering.

They had low quality motion vectors, and used FXAA as a fallback.
Yet it looks better than UE5 current option in real motion because all the ghosting you see disappears in 16ms and it has no weird smearing.

With UE5’s superiors motion vectors, and this “raw” frame could work very well with a SMAA fallback.
And that would still be cheaper than TSR, as I have tested. No smearing , just deletion of the motion vector lag.

Unreal needs this already, build on top of this idea. It’s been long overdue and you have the correct background more than anyone to implement it.

3 Likes
60

Is there a way to seperate static and movable object in TSR? I want to discard ghost-pixels of movable object that rendering on the static object.

1 Like
61

Deleted.

62

I’ve read a bit on that link you posted with the reddit thread with examples. And maybe I kind of understand what you mean ? But I’m still not sure.
I think there are two groups of people, and one of the groups would prefer absolute or maximum sharpness in spite of certain drawbacks like aliasing.

But I think for other people the ultimate goal of video games is to mimic reality. Or films. Most of devs companies look for the same thing. When some company posts a promo video game screenshot which looks way more realistic than other games before it, everyone goes :OMG, that looks so real, incredible… and so on. Same with promo video.
And if the goal is to mimic reality or films, if you take a screenshot from a film, you will see that lots of films look quite blurry in comparison with an aliased video game.

For example the company which makes Red cameras - professional cameras which are used in Hollywood films, slapped a sort of a blurry glass right before the sensor which captures images. It’s called an OLPF filter (Optical low-pass filter). And exactly like you, lots of buyers protested against that filter, and some opened the camera and took the filter out. And without that filter the image looks way sharper, but again - with aliasing drawbacks.
Just like how video games people are trying to mimic reality or cinema, the video camera people are trying very hard to mimic the film stock, the old reels of film made of plastic and photo chemical process. For everyone, that is the ultimate goal. The famous Christopher Nolan even today films with photochemical film. And if you look closely at a frame from a film, you will see it can look quite blurry as well.
And with that OLPF filter, sure, you can take it out, but people need to pick their poison. You want anti-aliased images which sort of mimic film stock ? Or you want ultra sharpness with aliasing and moire and other artefacts.
I think it’s the same in video game engines as well. A problem of choosing your poison.

2 Likes