TSR feedback thread

Hi!

Thank you for reading. Typos (If I missed any) will be fixed but I’ve been working on this post for several hours, recording, properly testing, and writing this.

Welcome and thanks for the genuine and extensive feedback, spending this time is well apreciated! Will do my best to be concise on verything said but please to not hesitate to bring up anything again I could have missed in my futur repplies.

(and no worries about typo and stuf, my english isn’t my native language and so it is far from being sheakspear quality :wink: , raw content I only what I personnaly care ).

TSR upscaling almost accommodates for the temporal blurring at least for more close objects. A custom depth-sharpen post process filter+ a camera blueprint that changes the sharpening values based on camera velocity can fix the controversial blurring effect.

So TSR developements in 5.1 was really focused on both PS5, XSX and XSS consoles and sadly shipped with an unfortunate overbluring issue in 5.1 especially on epic scalability settings.

That is because in few time dedicated for PC, an interesting discovery was made arround reprojecting frame N-2 into N to know how much it has been overblured when reprojecting into frame N-1 and then N. This is r.TSR.History.GrandReprojection. This new technic eliminated other technics use to counteract overblur at expense of image stability. But in attempts to optimising its runtime performance, it ended up loosing bit of its own quality. Good news in 5.2 is that it has been replaced with r.TSR.History.ScreenPercentage=200 while more time is being invested on this ( https://github.com/EpicGames/UnrealEngine/commit/9ccd56dfcc06852b5b89a81411972f81e3ac31e3 ) on epic scability settings that previously only on cinematic scalability settings. It’s already used in Fortnite Chapter 4 Season 2 and I’m really eager on community feedback on this change on 5.2. Sadly I’m failing to understand wheather your video are on 5.2 or 5.1.

This TSR change also exposed couples of new cvars to expose trade off between image blur or stability in reprojection notably with r.TSR.History.SampleCount, r.TSR.ShadingRejection.SampleCount, r.TSR.Velocity.WeightClampingSampleCount. As an example, Fortnite Chapter 4 Season 2 being competitive with a lot of shooting, is shipping with r.TSR.History.SampleCount=8.0 and r.TSR.Velocity.WeightClampingSampleCount=2 in the /Config/DefaultEngine.ini. If these looks intimidating to change TSR settings like this, I’ve added in 5.2 more extensive explanation of each TSR cvars, among with flush caveats and trade off each have in this change which I hope will make customizing TSR settings a lot more intuitive: https://github.com/EpicGames/UnrealEngine/commit/2d3e9b6e1b9e66dbe2807c9c80ee64987bc708f2 . Planning to have all these cvars and their detailed explanation exposed in TSR online documentation for 5.2 release or slightly after.

Naturally how much the image look statically can be hit different taste, whether it’s player test with setting often directly controlable on TV or monitor or content creator in their project. For instance Fortnite adds r.Tonemapper.Sharpen=0.5 on all platform using TAA, TSR or third party upscaler. Now interestingly with that r.TSR.History.ScreenPercentage=200 is could expose even better look sharpening directly in the TSRResolveHistory.usf that does the downsample from history resolution to display resolution using a Mitchell-Netraval spatial kernel. Right now the parameters are hard coded in the shader but we could envisions a futur where sharpening instead happens here. :melting_face:

As you can see in the video that light pole smears on the sky. This is only an example of a strange smearing bug in TSR that happens on any object.

That building disocclusion happening is very interesting case where there can be some improvements: TSR knew how the building look before the pole moved over. Problem currently is when TSR discard the history, it is forever because it only keeps its history for frame N-1. It also keep N-2 for r.TSR.History.GrandReprojection, but depending of the thickness of pole, this might still not enough: maybe TSR discarded the details of that building at N-6. I’ve been thinking a lot how mechanism for doing more general purpose history resurection (for the lack of a better name) but there is couples of performance problems to tackle:

  • detecting when this can be done cheaply;
  • selectively reproject that holder N-X data into N which this experimental r.TSR.History.GrandReprojection brought up a lot of new interesting possibility especially with the use of D3D Texture2DArray;
  • TSR history memory footprint: Keeping say the last X frame in memory to possibly reproject doesn’t scale well as your pole thinkness on screen could be.

Sadly how much smarter TSR becomes of that “history resurection” on disocclusion, it is still important to keep in mind there will still be disoccluded areas on screen that will be discovered the first time and have the quality tied there only to how fast details are accumulating at image macroscopic and microscopic details.

This is where I hope the new in 5.2’s stat tsr command will bring more understanding that there will always be a limit how much lower the resolution can be, how much ScreenPercentage can be lowered based of the display resolution and is the reason for instance the screen percentage on a XSS in Fortnite chapter 4 presenting at 1080p is higher than on PS5 and XSX presenting at 4k.

That is also why on Fortnite on PC I added that default Recommended setting in the Temporal Super Resolution configuration that suggest a screen percentage based of displayed resolution using FStaticResolutionFractionHeuristic.

	FStaticResolutionFractionHeuristic StaticHeuristic;

	StaticHeuristic.Settings.PullRunTimeRenderingSettings();
	StaticHeuristic.Settings.Mode = EScreenPercentageMode::BasedOnDisplayResolution;
	StaticHeuristic.TotalDisplayedPixelCount = ScreenWidth * ScreenHeight;

	float RecommendedResolutionFraction = StaticHeuristic.ResolveResolutionFraction();

Hoping to bring that successful new Recommended setting by default in Lyra for future release of the engine.

Hopefully this a bug being worked on in 5.2.

Please be aware 5.2 already branched of UE5/Main and is being stabilized for release and when that happen it means most often, current ongoing developement is future release for stability reason.

Like for instance when the camera is rotating around an object. The road smears on the characters face. NOT acceptable. I’m not dissing TSR or the people who made it, but these issues that must not be ignored for the sake of UE5, current gen games, and healthy maturity of TSR

Yeah I see what you mean, this has been known to happen. But my experience this has been mostly noticeable at low frame rate or resolution that is very low TSR feed (MP/s).

However through your video, not sure to understand your resolution or frame rate due to the various limitation of video and its encoding. This is my falt I should have said on my first message what was the extent of information I was meaning when saying “illustration”. Screenshot, videos are best way to show the visual artifact, but it is always important to add some additional information, such as display resolution, rendering resolution (which the VisualizeTemporalUpscaler show flag have in 5.2 conveniently), and frame rate. The stat tsr in 5.2 will be ideal to record and share video of TSR failure too because displaying all this information directly on the video.

Would you mind sharing these further details you recorded your video with?

Far away moving objects from a stationary camera, ghosting. This next video below shows this problem and needs work as well.

We ran into this issue many times during the developement of the Matrix Awakens: the way the cars are drawn in the distance is not the same as when they are closer. And that code path drawing the geometry that also draws motion vector can regress for various and sadly to many reason. Whether this is engine regression (which it looks like in 5.2 because I swear this was fine in the demo released on console) or project settings or material settings or missing PreviousFrameSwitch to feed previous frame’s value of animated material parameters in some WPO animations for instance.

Because we have been very tied to runtime performance, no time was spent in TSR to do optical flow analysis to try to ease developement burden of motion vector break for various reason. So this is where in the time we had in 5.2 was first spent on tools to more quickly diagnose this sort of problem. Just open up that show VisualizeTemporalUpscaler or show VisualizeMotionBlur or show VisualizeReprojection and this should show immediatly how the motion vector look. This is where the upcoming documation of TSR is also being actively worked on to better explains how to diagnose the core reason of this kind of artifact.

Doing optical flow in TSR we are already investing research on, for better reprojection of translucencies, reflections, fast moving shadow. It’s just a matter of finding a solution that works with largest amount of content variety and in performant maneur universally on existing plaftorms we support in the end!

Listen, I personally hate temporal stuff because the overall blur hurts my eyes, but TSR has a lot of potential and I want to stay positive by giving my feedback on it.
I’m no expert but I think I have done as much digging on UE5 Anti-Aliasing as any normal person could. The devs who worked on this are brilliant and are on a whole other level than I am.
I watched the 5.2 demo and was very disappointed to hear it ran on a 4090 and still looked blurry when editing the level(it was beyond the lack of motion vector creation when new assets are copied or moved into scene for the first time).

Thanks a lot for your time, it is very well apreciated! :grinning:

Another thing is the confusion around how certain games on PS5 produced by Epic Games or say a certain fighting game by ramdi camco looking less artifact’Ish on RNDA2 console architecture.
Maybe a presentation on how to optimize TSR would help devs become better at making crisper and less smudgy looking games on PC architecture (If that’s currently possible?).
Bad or good enough AA isn’t going to be good for UE5. It needs to be amazing, on par with its other features like lumen and nanite. Or else, it’s just a step backwards.
I hope to find UE5 looking great and clear by the time I move to digital game production.

Or better documentation? :innocent: You are definitly right and it’s entirely my bad here! TSR in 5.0 had many problems and documenting these problems was a lot of time not spent on significantly improving the tech instead that was dearly needed. A lot of these improvements in 5.1 and rest in 5.2 and this is where the focus has been on tools and documentation for 5.2 release. This is also the goal of that public thread between community and Epic, where questions can be asked and accurate technical response can be shared to everyone immediatly while we learn what is missing and need to update our online our doc.

1 Like