User Tag List

Page 1 of 2 12 LastLast
Results 1 to 40 of 43

Thread: Sharp Temporal AA

  1. #1
    0

    Sharp Temporal AA

    I tempered with TAA code and managed to make it sharp at the cost of some aliasing. Artefacts (overall blurriness, smearing, ghosting) are reduced, temporal stability kind of looks like with 8xMSAA and specular aliasing is still taken care of. (And all this by only changing one number in code.)

    ...(installation path)/Epic Games/4.14/Engine/Shaders/PostProcessTemporalCommon.usf
    under the 'COMPUTE BLEND AMOUNT' section I changed "BlendFinal = 0.04" to "BlendFinal = 0.25"
    and run the console command "r.TemporalAASamples 2"

    Moving foliage looks a lot better this way and everything is sharper.
    What I still can't find out is how to change the number of frames TAA blends together.
    Last edited by helluvamesh; 01-16-2017 at 08:21 AM.

  2. #2
    0
    Luminary
    Join Date
    Mar 2014
    Posts
    1,607
    Definitely needs some comparison images.

  3. #3
    0
    Quote Originally Posted by ZacD View Post
    Definitely needs some comparison images.
    TAA's artefacts are best seen in motion images can't show them.

    Edit: OK, ghosting can be seen in images.
    Last edited by helluvamesh; 01-16-2017 at 08:30 AM.

  4. #4
    0
    UE4 TAA use exponential blending so basically number of frame is infinite.(but older frames have infinite small weight.). So there are no simple number of frames control.
    https://de45xmedrsdbp.cloudfront.net/Resources/files/TemporalAA_small-59732822.pdf

  5. #5
    0
    Quote Originally Posted by helluvamesh View Post
    I tempered with TAA code and managed to make it sharp at the cost of some aliasing. Artefacts (overall blurriness, smearing, ghosting) are reduced, temporal stability kind of looks like with 8xMSAA and specular aliasing is still taken care of. (And all this by only changing one number in code.)

    Moving foliage looks a lot better this way and everything is sharper.
    It sounds awesome, but if you could show some comparison images it would definitely be great! Still images work great for showing how the overall bluriness is reduced. Here I made a comparison between TAA and MSAA:

    TAA: https://puu.sh/rQ6aR/2b88ba58a1.png
    MSAA: https://puu.sh/rQ6a3/c7b9f48965.png

    Even with it being a still image you see how superior MSAA is because you can read the text. So if you could take some images showing the comparison between TAA default and TAA with your change that would be awesome!

  6. #6
    0
    Comparison of blurring:
    original:
    Name:  TAA_original_3.png
Views: 4258
Size:  181.0 KB

    edited:
    Name:  TAA_edited_3.png
Views: 4245
Size:  173.1 KB

  7. #7
    0
    EDIT: Certainly looks sharper, but be interesting to see how it behaves with SSR / SSAO etc. I suspect sharper TAA is likely to introduce more AO / SSR artefacts.

    Reducing ghosting is probably the biggest way to improve TAA. Just reading through that presentation, and maybe scaling pixel samples by their scene depth vs current pixel scene depth would be a good way to circumvent that.

    Beyond my expertise though and probably pretty expensive..

  8. #8
    0
    SSAO seems to be fine. As for SSR, well, it doesn't like it. SSR vibrates a bit even when the camera is still, but it depends on resolution. It's not that noticable.
    Last edited by helluvamesh; 01-16-2017 at 11:07 AM.

  9. #9
    0
    I have tested it now.

    I noticed that without the change, switching between r.PostProcessAAQuality 3 and 6 did not change anything, so it looked same.
    With the change, switching between r.PostProcessAAQuality 3 and 6 is a big difference, I think r.PostProcessAAQuality 3 equals how it looks like without the change, so only r.PostProcessAAQuality 6 is affected. Thats a good way to see the difference on-the-fly while being in the game without recompiling that shader.

    I do not see a difference between r.TemporalAASamples 8 (thats the default I think) and the r.TemporalAASamples 2 you said that should be used.

    With the change, the TAA is definitely making it less blurry. But there is visible aliasing, and SSAO also becomes a lot more noisy. Still, modifying the BlendFinal allows for fine tuning this quite well. I think with a value of 0.15 I am quite happy. I have tested this in VR, and regarding the blurriness it looks like 10-20% more screen percentage I think. SSAO becomes noisy, but still acceptable with 0.15.

    So, definitely a very nice change, thanks @helluvamesh for finding that

  10. #10
    0
    Quote Originally Posted by John Alcatraz View Post
    I have tested it now.


    I do not see a difference between r.TemporalAASamples 8 (thats the default I think) and the r.TemporalAASamples 2 you said that should be used.
    With 'r.TemporalAASAmples 8' I can see heavy vibration (when the camera is still) on lengthy thin geometry (window frame) and some on masked materials (foliage), r.TemporalAASamles 2' solves this for me.

  11. #11
    0
    I've just seen that Epic added 3 new console commands for tweaking TAA in 4.15 preview 1 and one of them modifies the BlendFinal: r.TemporalAACurrentFrameWeight. So, now it can be set from within the editor without having to manually change PostProcessTemporalCommon.usf

  12. #12
    0
    Quote Originally Posted by helluvamesh View Post
    I've just seen that Epic added 3 new console commands for tweaking TAA in 4.15 preview 1 and one of them modifies the BlendFinal: r.TemporalAACurrentFrameWeight. So, now it can be set from within the editor without having to manually change PostProcessTemporalCommon.usf
    Interesting, thanks! So setting r.TemporalAACurrentFrameWeight to 0.25 equals the change you did in the source?

  13. #13
    0
    Quote Originally Posted by John Alcatraz View Post
    Interesting, thanks! So setting r.TemporalAACurrentFrameWeight to 0.25 equals the change you did in the source?
    It should. They changed the line "BlendFinal = 0.04" to "BlendFinal = CurrentFrameWeight".

    There's also 2 more new console commands r.TemporalAAFilterSize and r.TemporalAACatmullRom that I don't know what they do yet.

  14. #14
    0
    Samaritan
    Join Date
    May 2014
    Posts
    86
    Just using r.Tonemapper.Sharpen 1 or 2 also does a surprisingly good job reducing the blurriness without adding any sparkling. We shipped Ripcoil for Oculus using that.
    Last edited by OwenWP; 01-20-2017 at 03:25 PM.

  15. #15
    0
    Quote Originally Posted by OwenWP View Post
    Just using r.Tonemapper.Sharpen 1 or 2 also does a surprisingly good job reducing the blurriness without adding any sparkling. We shipped Ripcoil for Oculus using that.
    Thanks for mentioning that. I've tested it now and it seems to have roughly the same impact like modifying the BlendFinal variable.

    In my opinion modifying the BlendFinal variable looks a bit better because it introduces a less noticeable noisiness to the SSAO than the r.Tonemapper.Sharpen.

  16. #16
    0
    Samaritan
    Join Date
    May 2014
    Posts
    86
    Also it can be good to add a negative mipmap bias to your textures, using r.MipMapLODBias=-1 and
    r.Streaming.MipBias=-1. This will make normal maps in particular significantly sharper, and any other texture with lots of high frequency detail. Temporal AA gives the best image quality when it is given lots of different samples over time, and the "correct" mip map levels do the opposite.

    You do trade off a bit more smearing though and it increases your VRAM usage significantly, so consider putting it in your scalability settings, or only apply a bias for certain texture samplers in your materials.

  17. #17
    0
    looks a lot better in VR too!!

    more 3d definition.

    Thanks for the tip!

  18. #18
    0
    For me, 0.25 and 0.15 gives good result for grass at short range, but adds noticeable noise to all other objects in the distance.

    Wonder if anyone found a good combination of newly available settings.

  19. #19
    0
    Somewhat off topic. But a trick I use in games is this:

    You can set the screen percentage in World settings to 120% and it should produce a clearer result since the AA is then applied before down scaling to screen size. Adding a sharpening post process (10-20%) should also help make the scene sharper without causing aliasing.
    This will however use more resources since you render 20% more pixels.

  20. #20
    1
    Quote Originally Posted by hallatore View Post
    Somewhat off topic. But a trick I use in games is this:

    You can set the screen percentage in World settings to 120% and it should produce a clearer result since the AA is then applied before down scaling to screen size. Adding a sharpening post process (10-20%) should also help make the scene sharper without causing aliasing.
    This will however use more resources since you render 20% more pixels.
    Not sure I'd opt for a solution that makes the end-user render more pixels on screen than what is already necessary, especially in complicated scenes. I can see a few use-cases for this though!

    After experimenting with numerous setups, I think I may have found the diamond in the rough...maybe...

    UE4 4.15.1 - Temporal Anti-Aliasing Settings

    r.TemporalAACurrentFrameWeight .5 (A bit higher than what I've seen mentioned but anything less than this introduced some pretty unusual artifacting in the scenes I tested it on)
    r.TemporalAASamples 4 (5-6 Samples seemed to introduce quite a bit of vibration/stuttering. 4 looks B-E-A-U-TIFUL)
    r.Tonemapper.Sharpen .5 (One of the only downsides to TAA is the 'bluriness' but sharpening the render just a bit eliminated this effect for me)

    Both SSR and SSAO seem to be unaffected quality-wise using these settings.

    Keep in mind that these may not be optimal for all setups (but they were in just about every scene I threw at it). Also, on a side note, I've spent about 4-5 years in Unity but fairly recently switched over to UE4 (so take my advice with a grain of salt and do what works best for you!) I've just seen quite a few questions/concerns with TAA recently and figured I'd chime in and try to help!

  21. #21
    0
    This is absolutely the best single thing I've learned regarding visual quality in UE4. Thanks!!!
    Artheon VR Museum: http://artheon.co
    Impossibility Labs Inc.: http://impossibilitylabs.com

  22. #22
    0

    Post

    So, I'm finding those settings pretty brutal on my eyes (at least in a dynamic, deferred scene). Here's what I do:

    ConsoleVariables.ini
    Code:
    ; Sharpen on the tonemapper side.
    r.Tonemapper.Sharpen=1
    And since I don't really care about performance right now:
    Code:
    ; Max AA quality.
    r.PostProcessAAQuality=6
    r.TemporalAASamples=32
    r.TemporalAASharpness=1
    Post-Stack Sharpening Effect
    That gets me most of the way, but I also have a fairly simple sharpening post material that I apply in my post-stack as well. I just tossed them up in our public repo: https://github.com/joymachinegames/joymachine-public/tree/master/ue4

    I have the base material as well as my settings on a material instance of it in there.
    Trent Polack
    Founder & CEO of Joy Machine
    Steel Hunters, a third-person mech shooter with randomly-generated items and enormous enemies.
    @mittense | @joymachinegames

  23. #23
    0
    Quote Originally Posted by mittense View Post
    So, I'm finding those settings pretty brutal on my eyes (at least in a dynamic, deferred scene). Here's what I do:

    ConsoleVariables.ini
    Code:
    ; Sharpen on the tonemapper side.
    r.Tonemapper.Sharpen=1
    And since I don't really care about performance right now:
    Code:
    ; Max AA quality.
    r.PostProcessAAQuality=6
    r.TemporalAASamples=32
    r.TemporalAASharpness=1
    Post-Stack Sharpening Effect
    That gets me most of the way, but I also have a fairly simple sharpening post material that I apply in my post-stack as well. I just tossed them up in our public repo: https://github.com/joymachinegames/joymachine-public/tree/master/ue4

    I have the base material as well as my settings on a material instance of it in there.
    r.TemporalAASamples=32 does not affect performance but it just tell how many jitter locations temporalAA will use. Bigger the number more time it will take to full cycle to loop. This might cause judder and excessive blurriness. I have noticed that r.TemporalAASamples=4 or 8 is optimal depending on target frame rate.

  24. #24
    0
    Quote Originally Posted by Kalle-H View Post
    r.TemporalAASamples=32 does not affect performance but it just tell how many jitter locations temporalAA will use. Bigger the number more time it will take to full cycle to loop. This might cause judder and excessive blurriness. I have noticed that r.TemporalAASamples=4 or 8 is optimal depending on target frame rate.
    I don't know, after jumping back and forth between all the values from 1/2/4/8/16/32, I'm still of the mind that 32 looks the best. Here's my scene:

    Trent Polack
    Founder & CEO of Joy Machine
    Steel Hunters, a third-person mech shooter with randomly-generated items and enormous enemies.
    @mittense | @joymachinegames

  25. #25
    1
    I went with these settings. I think it might vary a bit from setup to setup.

    Code:
    r.TemporalAACurrentFrameWeight = .1
    r.TemporalAASamples = 4
    r.Tonemapper.Sharpen = .2
    Before/After images: https://imgsli.com/MDQxMg

  26. #26
    0
    Found combining the information here with the ghosting change here

    https://forums.unrealengine.com/showthread.php?121443-Character-Ghosting-with-Temporal-AA&p=745592

    TXAA is much better. Just thought I would post it for anyone that runs into this thread. With the 2 changes it is much more usable.

  27. #27
    0
    I played around with the settings some more and ended up with a pretty decent result. The sharpness is well preserved while avoiding both ghosting and aliasing when moving around.

    Name:  ghosting.png
Views: 333
Size:  593.5 KB

    Here is a default/adjusted example: https://imgsli.com/MDU3Mw
    And here is No AA vs. adjusted TAA: https://imgsli.com/MDU3MA (Notice how well the sharpness is preserved)

    Code:
    r.TemporalAACurrentFrameWeight 0.2
    r.TemporalAASamples 4
    r.Tonemapper.Sharpen 0.8 // Somewhere between 0.5 and 1.0 seems like a sweet spot.

    No AA vs. TAA (Notice how well the sharpness is preserved)


    Video showing No AA vs. TAA




    Default TAA vs. Tweaked TAA


    Reduce ghosting

    The biggest factor to ghosting seems to be Motion blur. Turning that off helps, but you can also do the following.

    Here is a trick I used to reduce ghosting even more by changing the last frame weight when the camera is moving.

    Name:  okSJlfz.png
Views: 364
Size:  169.1 KB



    Quote Originally Posted by Joshua Halls View Post
    Found combining the information here with the ghosting change here
    https://forums.unrealengine.com/showthread.php?121443-Character-Ghosting-with-Temporal-AA&p=745592
    TXAA is much better. Just thought I would post it for anyone that runs into this thread. With the 2 changes it is much more usable.
    Those changes was added by default in 4.15, so no need to play around with the source just for that.
    Last edited by hallatore; 08-12-2017 at 10:55 AM.

  28. #28
    0
    Quote Originally Posted by hallatore View Post
    I played around with the settings some more and ended up with a pretty decent result. The sharpness is well preserved while avoiding both ghosting and aliasing when moving around.

    Code:
    r.TemporalAACurrentFrameWeight 0.2
    r.TemporalAASamples 4
    r.Tonemapper.Sharpen 1 // Somewhere between 0.5 and 1.0 seems like a sweet spot.
    looks quite good, these are the exact settings I used for my UE4 summer jam entry (shootergame-based). in my eyes it makes the edges look slightly jagged but it's also the best tradeoff I could get

    however it's a case by case basis. in the project we have at work (top-down city builder) the ghosting it very extreme from fast camera movements. I think even for third person games it's also not enough


    even with these settings I think the biggest issue right now is how the Current Frame Weight works: it stacks on top of the previous frame which already stacked with the previous one and so on, so all each previous frame is still there, in lower opacity each time, up to infinity.
    if it could somehow work that it keeps the last X frames as buffers and average them alone (so older frames are already discarded) the temporality would be greatly reduced and so the ghosting would be minimized. however I don't think keeping X full-resolution buffer frames is really affordable
    Last edited by Chosker; 08-12-2017 at 03:39 AM.
    Help me back! follow me on Twitter

    Developer of Elium - Prison Escape

  29. #29
    1
    Quote Originally Posted by Chosker View Post
    even with these settings I think the biggest issue right now is how the Current Frame Weight works: it stacks on top of the previous frame which already stacked with the previous one and so on, so all each previous frame is still there, in lower opacity each time, up to infinity.
    if it could somehow work that it keeps the last X frames as buffers and average them alone (so older frames are already discarded) the temporality would be greatly reduced and so the ghosting would be minimized. however I don't think keeping X full-resolution buffer frames is really affordable
    I made a this to change the frame weight when the camera is moving. I think the result looks really good.
    Nice temporal stability when standing still, no ghosting when moving around.

    Still need to tweak the movement threshold and the weight amounts though.

    Last edited by hallatore; 08-12-2017 at 05:30 AM.

  30. #30
    0
    curious to see what you've made but the attachment isn't working
    Help me back! follow me on Twitter

    Developer of Elium - Prison Escape

  31. #31
    0
    Quote Originally Posted by Chosker View Post
    curious to see what you've made but the attachment isn't working
    Fixed it

  32. #32
    0
    Quote Originally Posted by Chosker View Post
    In the project we have at work (top-down city builder) the ghosting it very extreme from fast camera movements. I think even for third person games it's also not enough
    Here is a demo of the settings above. It's not so easy to see on youtube, but I didn't see any ghosting at all on my screen.


  33. #33
    1
    Tweaked the code some more. Sharpen set to 0.8 is pretty much the same sharpness as with no AA at all.
    I also updated the blueprint attached to the camera. It sets LastFrameWeight to 0.5 and reduces Sharpen to 0.5 (the higher LastFrameWeight, the less Sharpen we need)

    Here are 3 examples where I move the mouse back and forth with motion blur amount set to 0.1. Quite pleased with the result

    Name:  ghosting.png
Views: 334
Size:  593.5 KB


    Name:  ghosting_bp.png
Views: 331
Size:  166.1 KB

  34. #34
    0
    It is all cute, but
    Code:
    r.TemporalAACurrentFrameWeight
    Re-introduces projection matrix jitter into the image, and at values above 0.1 it is highly noticable.

    Something witty has to be done with history clamping. Has anyone actually tried something in that field? Maybe some sort of clamping algo, that is adaptive to velocity ?

  35. #35
    0
    Quote Originally Posted by Deathrey View Post
    It is all cute, but
    Code:
    r.TemporalAACurrentFrameWeight
    Re-introduces projection matrix jitter into the image, and at values above 0.1 it is highly noticable.

    Something witty has to be done with history clamping. Has anyone actually tried something in that field? Maybe some sort of clamping algo, that is adaptive to velocity ?
    I have almost no jitter with the settings above. But if I set r.TemporalAASamples higher than 4 they start to show.
    Setting r.TemporalAACurrentFrameWeight to 0.2 was originally to avoid ghosting. But with the camera tweak you could possibly get away with setting it to something like 0.1.
    Also it seems motion blur and TAA goes hand in hand. You should probably use different based on motion blur on/off.

    Preferably I would set the TemporalAACurrentFrameWeight per pixel based on it's motion vector. But I have no idea how to start playing around with that.

  36. #36
    0
    Quote Originally Posted by Deathrey View Post
    Something witty has to be done with history clamping. Has anyone actually tried something in that field? Maybe some sort of clamping algo, that is adaptive to velocity ?
    This causes quite a bit of console log spam, but it seems to actually work for third person camera.

    Name:  ghosting_lerp.png
Views: 325
Size:  176.3 KB

  37. #37
    0
    With the camera logic I posted above I have tweaked the settings a bit.

    The biggest difference is that the texture sharpness now is exactly the same as with no AA applied. (Compare NO AA vs. these settings: https://imgsli.com/MDU3NQ)
    Here you can compare the default AA's sharpness vs. these settings: https://imgsli.com/MDU3Ng


    And here you can see the temporal stability is maintained.




    New settings

    Code:
    r.TemporalAACurrentFrameWeight 0.1
    r.TemporalAASamples 4
    r.Tonemapper.Sharpen 0.5

    New camera adjustment logic
    I tweaked the blueprint to adjust the settings based on camera movement. Combined with these settings it seems to work very well.

    Name:  ghosting_v2_bp.png
Views: 318
Size:  174.3 KB

    You can copy & paste this into your blueprint to get the code: https://gist.github.com/hallatore/ce22e268e1ec9f376b423b582ae7d48b
    PS: I've set the max FrameWeight to 0.35. This seems to be a good balance between temporal stability and ghosting.
    Last edited by hallatore; 08-12-2017 at 12:58 PM.

  38. #38
    0
    Samaritan
    Join Date
    Nov 2015
    Posts
    124
    Quote Originally Posted by Deathrey View Post
    Maybe some sort of clamping algo, that is adaptive to velocity ?
    But only if you entertain specular aliasing, and some cheeky noise once you turn your camera around :P

  39. #39
    0
    Well, I'm not a big fan of using Tonemapper sharpen to solve ghosting issues. It introduces artifacting of its own.
    Tweaking current frame weight based on world speed of the camera might seem situationally useful for certain games. You can also scale final blend by velocity data in the shader.

    Problem with using reduced samples for TAA, is that at 4 samples, you are just using typical MSAA jitter offset pattern. It reduces TAA effectiveness to some degree as opposed to Halton sequence at 5 samples and above.

    Improving TAA ,in my view, should be probably separated into two parts. The first part is coming up with better way of history clamping(I believe that I've read somewhere that orienting and stretching neightbour sampling pattern in direction of velocity helps, but haven't tried it.).
    The second part would be a better solution to calculating velocity offset based on depth difference. Right now the cross pattern with offset of 2 is used used by default. I've actually had some limited success with using a larger SNN filter for that, which blurs the borders of objects more, but results in less overall smearing of multiple layers of moving objects. This one in particular relates to foliage.

  40. #40
    0
    Quote Originally Posted by Deathrey View Post
    Well, I'm not a big fan of using Tonemapper sharpen to solve ghosting issues. It introduces artifacting of its own.
    Tweaking current frame weight based on world speed of the camera might seem situationally useful for certain games. You can also scale final blend by velocity data in the shader.

    Problem with using reduced samples for TAA, is that at 4 samples, you are just using typical MSAA jitter offset pattern. It reduces TAA effectiveness to some degree as opposed to Halton sequence at 5 samples and above.

    Improving TAA ,in my view, should be probably separated into two parts. The first part is coming up with better way of history clamping(I believe that I've read somewhere that orienting and stretching neightbour sampling pattern in direction of velocity helps, but haven't tried it.).
    The second part would be a better solution to calculating velocity offset based on depth difference. Right now the cross pattern with offset of 2 is used used by default. I've actually had some limited success with using a larger SNN filter for that, which blurs the borders of objects more, but results in less overall smearing of multiple layers of moving objects. This one in particular relates to foliage.
    The sharpen is to "re-sharpen" the image since blur is a side effect of TAA.

    It seems there are two scenarios. With and without motion blur. And I can only seem to produce ghosting with motion blur enabled.
    The good thing is that it seems without motion blur higher sample count and lower FrameWeight works nicely.

    So I see it as two separate problems.

    1. TAA introduces blur. Using sharpen at 0.5 seems to nullify that. (if you sharpen without TAA you would of course over-sharpen the image)
    2. Camera Motion blur introduces ghosting.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •