Gen 5 Temporal Anti-Aliasing

Couldn’t get that fix in 4.26.2 because invalidating too many shaders for a minor release, but you can cherrypick that change: https://github.com/EpicGames/UnrealEngine/commit/8631cb027cef93f1e59d0ea11a6fdcce041f3aca

I found that the issues I saw with ghosting on foliage was caused by the Speedtree Wind which was applied as WPO. If I replaced the speedtree wind with some custom wind calculation in WPO, there no longer was any ghosting with Gen5 TAA. It seems the Speedtree Wind somehow does not work with Gen5 TAA. It works nicely with Gen4 TAA, but with Gen5 TAA, it always resulted in that very strong ghosting. It’s no longer an issue for me now though, as I just stopped using Speedtree Wind and replaced it with custom wind, as that also has other benefits.

I also managed to fix the ghosting on lit translucent particles by enabling the “Responsive AA” option for them. For some reason Gen4 TAA does not need that on them, but with Gen5 TAA, it’s absolutely required to not have very strong ghosting.

Ah, too bad! Would love to have those improvements in 4.26, as I’m not sure if I’ll ever be able to update to UE5. Having a well working Gen5 TAA would be a huge visual improvement for me.

I have basically done all my tests with a screen percentage of 50, comparing Gen4 TAA and Gen5 TAA. In my tests Gen5 TAA is usually roughly as expensive on 50% SS as Gen4 TAA is at 70% SS. So I’ve spent quite some time comparing which of those options looks better overall.

The main issue I noticed with the Gen5 TAA now after more testing, and after I fixed the foliage and particle ghosting issue, is that Gen5 TAA looks bad when the player moves (walks) around. The image with Gen4 TAA always looks pretty much the same, no matter if the camera is static, or moving. Gen5 TAA looks really good when the camera is static, or only rotating. But when the camera moves, the image quality starts to fall apart, and there are a lot of areas in the image which look as if there would be no AA at all, so very pixelated. What’s somewhat weird is that it’s not happening on the whole image uniformly, instead there are “blotches of pixelated areas” on the image when the player moves. These blotches seem to be somewhat local to specific screenspace areas. Does that make sense? Why does the image quality not uniformly become worse? Why does rotating the camera around (Yaw/Roll/Pitch) look perfect, but moving it (X/Y/Z) look bad?

Due to this issue, Gen4 TAA overall unfortunately looks better as it has more consistency in the image. Gen5 TAA is superior when the player stands still and only looks around, Gen4 TAA is superior when the player moves. And the fact that there is no significant difference in image quality between static/moving camera with Gen4 TAA makes looking at that more pleasing, while the constantly changing quality of the Gen5 TAA feels bad.

Is that deterioration of the image specifically while the camera moves expected with the Gen5 TAA?

If there are improvements that could be cherry picked, but as just too “dangerous” to be added to the 4.26 branch, would it maybe be possible for you to still make those accessible to be cherry picked by people who want to experiment with trying them out, maybe by just creating a PR to 4.26 for each of those improvements? I’d love to try out any such changes if they might improve the image quality of performance of the Gen5 TAA in 4.26. It really feels like not much is missing for this to be an amazing feature, if I could somehow get the image quality with a moving camera more similar to a static camera (or at least as good as with Gen4 TAA), and maybe slightly improve the performance, I could make Gen5 TAA the default option in my game, and that would hugely improve the visual quality.

Thanks very much for your great work on this!

2 Likes

Hi John,

based on what you describe, the core issue is that the SpeedTree’s calculation for the motion vector is wrong that is causing Gen5 TAA to ghost. We haven’t investigated that issue. :frowning:

So here Gen5 TAA have quite concerning ghosting issue when the particle becomes too small on screen. This is high on my list to tackle, but I still don’t have a solution that doesn’t kill performance yet. :frowning: When the responsive AA is enabled, Gen5 TAA actually completly behave identically to Gen4 TAA with its own trade off.

Another alternative has been to enable this translucent particle to draw velocities in the material settings and relis on Gen5’s TAA motion vector based parrallax disocclusion heuristic. However we have found that the motion vector for niagara particle where not as accurate as we would have liked, and we have been investigating what solution we will have to have accurate motion vector. Stay tunned.

Oops that is interesting, are you targetting 120hz? What GPU you have? Have a ProfileGPU log output to share?

I’ve never seen that artifact before. I wrote Gen5 TAA initially using Lumen in the Land of Nanite demo that have a character walking too before cherry picking this work in 4.26 for Fortnite on next gen consoles. Fortnite uses characters too and isn’t running into such issue as well.

To me this sounds like another motion vector problem in the skeletal mesh code path you are specifically using. The visualize motion vector I posted earlier might help you indentify if this is the issue. Feels free to post screenshots you feel confortable sharing. Maybe I’m wrong and there is a bigger problem for the particular content you are trying to build. :wink:

I preffer to stay honest and transparent and please apologises for any disapointment: I don’t know if I’ll have the bandwidth myself to maintain a 4.26 pull request with all the lattest of Gen5TAA in it. The challenge of TAA is that all the pixel of renderer goes through it, and this implies pretty high standard on quality and performance needs for Gen5 TAA to successfully ship in production. Due to the amount of work generated by this goal, we indeed have a higher priority for me to focus on improving quality and performance. Moreover we are also dog fooding it on internal project too where I also need to help when something goes wrong. :o

Having said that, we find important and I personally like taking the time to throroughly help you guys out on this forum thread with detailed response to as many question you all may have, because this is centralized knowledge any user of current and futur version of Gen5 TAA might also be please to find to understand and fix the issues they might be running into that ultimately is also part of my job to make the transition to newer TAA as smooth as possible for everyone. This also helps me making sure feedback about this functionality remains positive and there isn’t any huge problem I could have missed. :rolleyes:

While some UE5 improvement to Gen5 TAA are still not publicly available on github at this day, I would personnaly love to comment further details than I already have, but sorry I don’t think I can just yet. :rolleyes:

One thing I can comment though is it is still under heavy developement, and given the variety amount of content rendering through TAA, a regression on specific content case can happen, and might not be ideal as an user experienced. TAA is a delicate hand crafted heuristic to decide whether to use previously rendered frame or not. Gen5 TAA is just a steroid boosted (and slower :p) crafted heuristic. Hand crafted because this is the only thing we can do to work on maximum number of high end platforms that may not have expensive machine learning hardware. However not every user have the time or the knowledge to go tune an heuristic as complicated as Gen5 TAA for his content. So my goal is to make a TAA that just works (assuming there isn’t yet again wrong motion vector rendered somewhere for some pixel :p). The problem of hand crafting means decision needs to be made over what artifact should be prioritised over others. And this heavy development may also need to have some tradeoff between some artifact versus the others to also be reevaluated. And I know this as a user might be very frustrating because it used to work well before, but not anymore in his unfortunate very specific case. All that to say, what we have release in 4.26 might not be the lattest sadly, but it might not be too bad. :cool: Despite 4.26’s is shipping at production scale on Fortnite, it is still an experimental tech that may change.

Thanks for the info!

Yes, I am targeting and have tested it with 120 hz. Tested with both a RTX 2070 Super and a RTX 3090, the ratio of how much slower Gen5 is compared to Gen4 seems to be roughly the same. I till now mostly only compared total GPU frametime, not specific TAA time, so I don’t have exact profiling for those. I don’t want to spend too much time with profiling it now though before I found some way to fix the pixelation/aliasing issue, because before that’s fixed, the performance doesn’t actually matter too much as it’s not really usable due to visuals anyways.

The issue can’t be related to any skeletal meshes, I am not using any skeletal meshes. The player camera is just a floating camera, my player neither is a ACharacter (it’s only a APawn), nor has a mesh.

There are very clearly visible blotches of aliasing/pixelation traveling across the image when the camera is moved. Not when the camera is rotated or static. Only when the camera is moved. And Gen4 TAA does not have that issue.

I can’t post a screenshot publicly, but I’ve sent you a short youtube video of a few seconds of screen recording in a PM where you can clearly see those blotches of pixelation traveling across the screen when the camera is moved. The issue is same no matter if R11G11B10History is enabled or disabled.

That sounds like it might be possible to tweak some parameters to more ideally match the TAA performance to our own content though, if we wanted to spend some hours with fine tuning parameters for our specific content. I’d definitely be willing to do that if it means getting a better result at the end. Having something that works well out of the box is cool, but fine-tuning parameters also is not too big of a deal. Would it maybe be possible, for those people who want to do it, to allow fine-tuning such parameters with some console variables, or maybe even just shader constants that can be adjusted in the TAA .usf or .ush files? If changing those does not trigger a whole engine shader recompile, that sounds easy to adjust and tweak.

How much stuff is hardcoded to the view matrix jitter being within a single input pixel’s span? I’d ultimately like to use temporal upsampling in combination with variable rate shading, for VR, and that would require jittering within the span of the largest VRS block of pixels (probably 4x4).

I tried just modifying the jitter span from -0.5/+0.5 to -4.0/+4.0 to see and it seems ok for gen4 and gen5.

I didn’t try mixing in VRS yet and that would also need other changes to make the reconstruction aware of it. I think the Halton sequence may also need to be changed to be multi-octave as well.

Side issue: with no engine changes there seems to be a bug with r.TemporalAASamples. If increased much beyond the default 8 (I noticed it starting at 12) it will periodically run with what looks like no jittering and a lot of aliasing accumulation, and then return to jittering and look fine, over and over. Also this ensure gets tripped:



ensureMsgf(TemporalJitterIndex >= 0 && TemporalJitterIndex < TemporalJitterSequenceLength,
TEXT("TemporalJitterIndex = %i is invalid (TemporalJitterSequenceLength = %i)"), TemporalJitterIndex, TemporalJitterSequenceLength);


And it seems TemporalJitterIndex goes negative. Seems the Halton function will always return 0 when it is given a negative index and that probably causes the oscillation between aliased/unaliased.

edit: Ah, it seems to only happen at low screenpercentages, and so be due to this line boosting the sample count beyond int8 positive range:



// When doing TAA upsample with screen percentage < 100%, we need extra temporal samples to have a
// constant temporal sample density for final output pixels to avoid output pixel aligned converging issues.
TemporalAASamples = float(TemporalAASamples) * FMath::Max(1.f, 1.f / (EffectivePrimaryResolutionFraction * EffectivePrimaryResolutionFraction));


(TemporalJitterSequenceLength gets set from that)

This line should instead clamp to 127, since it gets put into an int8 at some point:



                TemporalAASamples = FMath::Clamp(TemporalAASamples, 1, 255);


I got some decent results. I’m just testing with individual meshes using VRS now because I couldn’t get the whole screen or the texture based one to work yet.

The chair on the left is using 2x2 VRS, and the one on the right normal 1x1. I added an additional mip bias of -1 since when using TAAU it is being sparsely sampled over many frames.

It seems to get significantly less aliasing with the 2 pixel TAA span, but still has some moving noise which I think would be able to be cleared up by making the reconstruction filter aware of VRS.

I think this kind of thing could benefit both this and nvidia’s DLSS with VR. The reconstruction itself could also output to a lower resolution too using VRS or its own logic, so that the texture bandwidth savings aren’t lost from the extra mip bias (for VR the periphery gets extra downsampling anyway due to lens warp).

In each image the chair on the left is using 2x2 variable rate shading.

TAAU (1 pixel jitter span):

Modified TAAU with 2 pixel span:

No AA:

Both Gen4 and Gen5 handled it similarly, with similar improvements with the increased span, I can’t remember which was screenshotted.

When using 4x4 VRS a changed pixel span up to 4 pixels made an even bigger improvement over the default span, but it started getting stronger moving noise and really needs adjustments at that point to not have the 4x4 pixel block edges and corners be seen as underlying features to reconstruct from.

4x4 VRS shows the improvement more. I made sure all these with aa were with gen5 turned on.

Default gen5 90% screenpercentage:

gen5 modified with a 4 pixel span to the jitter pattern, 90% screenpercentage:

no anti aliasing, 100% screenpercentage:

I added a pull request here to fix the sample count overflow issue with temporal aa jitter (more likely for people to run into it with TAAU at low screen percentages, because the configured sample count gets amplified there):

https://github.com/EpicGames/UnrealEngine/pull/7903

1 Like

I’d just like to say your work means a lot. And Communicating with us here is really rare from UE developers, no offense intended. It’s not like other engines, hard to find devs that give time to communicate with us here. Really appreciated.

Aliasing/ghosting is one of the biggest problems I face with dense vegetation in UE, anyone that improves anti aliasing is a legend in my eyes – so thank you.

Upgrading to UE5 seems to be fairly simple, so I’m really looking forward to the temporal improvements coming with UE5.

@Guillaume.Abadie
Are directives

r.TemporalAASamples
r.TemporalAACurrentFrameWeight

Have no effect when Gen5 TAA enabled in 4.*? I’ve tested multiple combinations and astronomical values there is 0 perceived difference so I assume they dont come into play?

2 Likes

Almost giving up on UE cos this issue. All foliage is rendering like garbage, AA seems to get things looking even worst.

Read the thread, proper motion vectors support been stressed out multiple times for tempAA.

@Guillaume.Abadie Thanks for some explanation in the thread. Little wonder if there’s any available paper/report about the optimization of Gen5 TSR, for either ongoing or completed version, compared to previous UE4 TAAU. Pretty interested about that

I was late to discover this, the next gen TAA solution has made me a believer again. I am amazed at how good this is, this is what I needed. I’m now sporting 50% screen resolution and I can barely tell the difference (and I almost always can).

Thank you for your brilliant work <3

@Vioxtar

Are you talking about the same AA as in this thread or is there a newer solution?

Perhaps the op of that thread didn’t tweak it enough?
But if we’re talking about the same AA as was shown in the Matrix demo, hoo boy, that wasn’t pretty… I mean everything else was spectacular, but the temporal smearing just grinds my gears so hard.

Ghosting was addressed by Gulliame earlier:

For jitter/instability at still image, you need to tweak TAA SampleCount and FrameWeight.
Either way when framepace is unstable and FPS spikes between crazy high and crazy low values - ghosting is inevitable due to nature of TAA

Yea, there’s a conflict between SpeedTree9 and Gen5 TAA, in UE5 this resulted in obvious ghosting and smearing, the SpeedTree support team told me to turn on “Output velocities due to vertex deformation” in the project settings, but it was useless, I rely heavily on speedtree, so I hope that solves the problem.
Anyway, I’m very grateful to have EPIC technicians here to answer questions, so thank you!

2 Likes

can i ask what is difference between gen4 and gen5 anti-aliasing and what does that mean??

I see Unreal 5 has something called TSR. How does that play into what’s been said in this thread?

Are there any updated preferences for anti-aliasing when it comes to VR?