About
First and foremost i will state that nothing is wrong with Epic’s TAA. You disagree with me, and that’s okay, i too am can’t find it exceptionally flawless but it is a very good implementation from Epic, and they have took great care of the code over numerous iterations in the last couple of years to make it work better for you. It’s something you can depend on, a friend that always work out for any of your situations, but not without the side effects. It ain’t perfect. It cannot be, but Epic have gave us a very good, alias free nice looking - almost photo realistic outputs for our joys. They did a very good job on this too, and i only tampering with it’s code because it does not always suit to all my needs.
Neither does fit yours. Right? That’s why you are here, so let’s get to it. Hereby in this video i will present you what can be the results of a tiny little change of the code, that will not only hide many of the well know problems, but makes your scenes look better (relatively speaking) without giving up too much on the stability and features. Yes, i’m just gonna hide the problems. It’s no solution to the problems itself, but we will just pick up the broom from mommys corner and pull everything under the carpet. That will happen here.
The video showcase
There will be 4 takes in the video, and they all showing the same results but with different contents to display some of the issues and results. I will switch between the settings so always look at my hands!
1st scene is the noise demo, a kind of sadistic-synthetic pleasure for your torments (and i want to show you youtubes flaws in the video quality as well). The noisy environment pronounces the best how TAA leaves most of your scenes with strong and ugly ghost remainders behind the moving objects, burning in the previous history information and they just won’t disappear. By default this blending happens over the period of 25+ frames, but to make them to take longer i will set the scene frame rate to be 15 fps, so it will take about 2 seconds for them burn-ins to kind of disappear. With or without motion blur, they remain to be there, for a veeeery looong period of time.
Then i apply this magic console command i just implemented, called “r.TemporalAADynamicMode 1” which have a float input value of 1.0, and it will ignite the extra code added deep in the Temporal Anti-Aliasing shader to apply this adaptive blending on everywhere.
But what is the problem here with TAA? I can’t tell. I just made it disappear with a neat little trick. What basically will happen is that the wandering (moving) pixels will no longer burning in, but the blending weight will be set to be relatively small for all those moving pixels, so the antialias solution will be applied relatively weak on their remainders. They are moving, so they look jaggied and smeared already. I just remove them pesky bugs from the rest of the scene, and in turn the entire scene will become clean and sharp. Since it is based all on a quickly extrapolated velocity of the moving pixels (in screen space), therefore the static objects will retain their nice shape with the good amount of antialiasings on there. It will only affect the moving content. Really. Just see this for yourself. The spooky ghosts are gone, and the noise is less blurry, while the objects in the static background are nicely antialiased. The scene is now sharp and responsive, where it must be. Stable, smooth and shiny where it have to be. Cool balance, huh?
So yeah, thats enough of the noise tests, lets move to the Scene 2 (at 2:15) to the cherry trees from the free content of Infinity Blade: Grass Lands. It’s a cool package and i’m sure you’re familiar with it’s content, so there it is. Once i apply the adaptive mode, the trees and grass start to show some sharp and clean details, the flowers and leaves appears without this median blurred nightmare. I take a closer peek at a leaf and you will see it’s inner detail will no longer glued to the screen, but it moves nicely, once i take this magic velocity information into consideration at the pixel blendings.
Third scene (at 4:10), is the car scene. I will take a little time here to show you the problems with SSR as it is getting smeared all over the place, the TAA is also distorting it by bending them towards your viewport, and it can best seen on the reflections of the hills and the loop. Also the ghost shape is there because the road has became way too bright, and despite the brilliant Dynamic Anti Ghost solution of the Epic’s TAA, it will present an another problem on it’s own, by cutting those shadow areas on the bright and shiny road, hence the ghost shape appears. I also took a peek inside the car you can see its hull is burned into the road areas as well causing this ghost effect appear again.
Why don’t we make that disappear too, along with the distortions and everything? Poof. Magic. It’s gone. But here comes the expenses. The price of all this fancy trick. Did you know that those ugly smeared pixels you all hate are actually hiding something else? Specular Aliasing! Boom. There you will get a loads of it, since they are no longer hidden. This will be something you have to address later, but for now it will only show up under the worst circumstances, and they are only there for a moment while the shiny-thing is moving relative to the viewport. Once the movement will stop, the specular aliasing gets blended in again, and the problem is gone. Phew! Almost gone. Once you turn off this adaptive thingamajiggy, you will see the TAA is doesn’t quite takes care of the specular aliasing, as they usually pops up at the edges of the screen while you rotate or move the camera. Only for a moment, but they instantly gets smeared over in the next frames so they disappear. We can’t have this on the road, since they look awful. We want the specular aliasing there, thus showing those blazing fast speeds of our cars. Or maybe not?
Anyways, the specular aliasing i’m quite certain can be treated by many ways using other dark practices, like i could go with a little tinkering at the blending equation to exclude these high contrast areas. It probably would work out with HDR too, i did not tested this all. But let’s just leave something for the next day. Today is the big triumph of the brooms and carpets.
The final take (at 8:16) will be the jungle scene, it’s for your eyes and pleasure and because it looks very cool, i just love this scene from Dokyo. The leaves will be smeared first because the adaptive mode is always off, but once i enable it they became sharp and detailed. I also apply one extra sharpening pass on them which actually makes them a little jaggied and aliased, but oh well, youtube and it’s videos, i wasn’t sure how will it look at the end. (Likely it wont even show you half the fine details).
The how
Well you can take a good tour for yourself on how to modify the engine, and it worked out for me better to implement this console variable, so you can go that path too. But you don’t really have to and you might as well can just modify the PostProcessTemporalCommon.ush that will be enough. I will show you how to implement this stuff into 4.17 shaders but it can be done in other engine versions as well - it depends on which version you use, and might require some engineering (coder) work to figure out the right location and the implementation. But i’m quite certain there are ways to be added to both the Epic and Nvidia branches from very early versions to the future releases.
Here is the file for 4.17 and i just marked the line 566 for you where i would put this stuff:
https://github.com/EpicGames/UnrealEngine/blob/release/Engine/Shaders/Private/PostProcessTemporalCommon.ush#L566
Let’s just add this one single line of code.
BlendFinal = clamp( BlendFinal + sqrt(sqrt(distance(BackN.xy, UV.xy))) * 1.0f , BlendFinal , 1.0f );
Once you’re done, save the file, and go back to your engine. Type in the console the “r.ShaderDevelopmentMode 1” (without the quotes) and press the enter. It will make you feel safe and everything. Now you can press the magic key combo that will recompile the shaders and makes these changes final for you.
Ctrl+Shift+. (dot). The compile process will start and finish in a moment. In case you happen to be missed this line and made a typo, the compiler will warn you that you should fix the code before continue. Do it, save it and then you can press the Retry button to try again and see what you did.
That’s all, test your scenes and much joy. I maybe propose this one for a pull request (it will unlikely make it to the engine), but i’m more eager to hear about your ideas regarding this and other changes/improvements. It’s okay if you find this code is awful, i know it is. That’s the best i can provide for today. For any other cases, please enjoy and profit!
Tips
You can modify the value at the velocity (* 1.0f) to be larger or smaller, to apply the effects more pronounced. I just rigged my console command to that value, and i use 2.0f sometimes in the video.
This value have effects on the specular aliasing as well. Smaller the value, less the specular issue will be visible, but more the ghosts will peekaboo. You can also add or remove sqrt()'s around this velocity stuff to better adjust to your scenes needs, the fourth square i use here kind of works out on most of my sceneries, but a single square root can be more sensitive to the movements so it might suit better for the slow moving scenes. I also have experienced better clarity with values around 1.5f but that maybe just me.
Other Tips
Some forum members have came up with settings for the Temporal AA parameters to change the behavior, reduce/extend the number of samples taken and the blend weight. Those changes can be used in harmony with this modification, but they may present different results so you have to adjust the numbers accordingly. Just blame me if you have troubles with that settings. It’s all on me.
Improvements for the Next Gens
Since this modification does not resolve any of the problems but just hides some of them under the carpet, it is fair to assume there are much better ways to approach any of these problems and find good solutions to them. I’m also reading the forums and have seen a few members have took little notes and side mentions of certain improvements here and there, which actually would be very nice of them to share in greater details with others to learn from. The way the engine is approaching the temporal anti-aliasing along the rest of the equations are not the only parts require more attention but the actual behavior perhaps go certain ways that is most definitely won’t apply to many users.
Now after this little training you are ready to open up the code again, check it’s content and you can learn a great deal of lecture from there, thanks not only the comments they put in there, but the simple writing of the code itself. I highly advise you to think about what you see in there, and you will just come up with great many ideas i’m sure of it. The developers at Epic are usually following the ways of quick and simple paths, to put litte stress on the cpu/gpu while sacrificing little of the quality. But they make sacrifices. Experiments are there too. You can find many corners to improve on the quality!
It’s very easy to iterate on the shaders, and it have became fast to recompile them so it’s not that big of a deal, really.
The sharpen
(I’m not sure i must address this, but i leave this here for now.)
No you should not apply sharpen to any of your scenes. It will remove the nice antialiasing and leaves you off with grandma’s staircase at the old crackling house, on all the edges. Spooky stuff indeed! But i can’t tell you, what you do. You will just go with the sharpening. Well, here is an idea, which i put in the postprocess after the tonemapper.
That’s kind of an adaptive sharpening at a simple form. This will exclude the edges, but apply the sharpening only on the fine molten details. You will make it to be distance aware, and only sharpen the close up content by using the depth information of your viewport. The close objects will become more pronounced, but if you overdo you will end up with a shading draw. Just don’t go there, it’s enough to apply some fine amounts of it.
There also is a tonemapper sharpening console command, nice implementation, you can try it; it will pronounce the edges and removes some antialiasing as well. No, i don’t recommend it.
Final words
Thanks for your attention. Any questions? Additions? Please share all your ideas how else you would improve on the TAA!