Announcement

Collapse
No announcement yet.

Ray Tracing is beyond broken in 4.23

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

    #16
    Hi,

    We have been reviewing most of the issues discussed here, checking if there are regressions, and if we need to take immediate action. So far we have not found any issue introduced in Preview 8 but we are going to keep looking into skylight in reflections and any other topic discussed here to ensure bugs found are fixed as soon as possible.

    There is a second category of issues mentioned here that are not actual bugs but known limitations that need some more time to improve. Namely improving performance with many instances (foliage) or extending denoisers with the ability to support more than one sample per pixel are in this second group. We are working on these areas and will hopefully get good results here in the near future, but these are not things that can be improved in a few days.

    Besides, there are other topics mentioned in this thread that are open problems that do not have a clear solution in real time ray tracing yet. Since DXR compatible hardware was released last year and real time ray tracing techniques started to spread out, I have observed some confusion regarding what people expect from real time ray tracing. Real time ray tracing is not real time path tracing. In real time ray tracing we can trace a very low amount of rays per pixel (5-ish), which allows to solve certain problems pretty successfully (such as area shadows or reflections in cases where screen space reflections do not work) but it cannot solve at 30fps the same problems offline renderers solve tracing thousands of rays per pixel and taking minutes or hours to clean a frame. Results obtained with real time ray tracing are amazing in many cases, but a whole replacement of offline techniques will take time to happen.

    Specifically, problems such seeing ray tracing effects in reflections or refractions are very hard to solve (i,e area shadows in reflections, GI in translucency, etc) These problems cannot be solved in real time yet, we need more research, more novel ideas and eventually faster hardware.

    We will eventually get there. Ray tracing adoption in the film space was quite slow in 2005 because of the limitations of the algorithms at that time, but ten years later it was almost impossible to find a single frame in a movie that was not generated with ray tracing techniques. It will take time until the same happens in games, but some techniques such shadows already show great performance in many scenarios and it looks like this will become the de facto technique for shadows very soon, at least in desktop machines. Other techniques will follow later.

    I also understand some people are not interested in ray tracing yet, and will forward your suggestion of creating a separate subforum. With regards to this topic, do not be concerned about Epic prioritizing ray tracing on top of everything else. There are several major initiatives in the rendering team these days and ray tracing is one of them but there are also others. The rendering development branch is available in Github you so can check the activity in many non ray tracing topics is quite intense (and there are so many cool things in the works ).
    Originally posted by Rawalanche View Post
    Since you have the path tracer implementation, it should not be hard to test and verify if ray tracing components are providing correct results. I am very surprised so many weird quirks have gotten through under supervision of someone who has already done one commercial offline renderer, especially one where accuracy was the main priority .
    In some cases we might have introduced bugs. That can always happen, specially if the technology is still in beta. In other cases like reflections he issue is more that given the very low ray budget we need to make difficult decisions on when to trace more rays and when to clip or fall back to other mechanism. Ideally we should always provide raytracing passes that when increasing the amount of samples and the quality knows they converge to the output given by the path tracer, but this is not always possible because of a number of reasons (some of them related to performance and others to the shenanigans of splitting the rendering equation into separated ray tracing passes). In any case we are very thankful for the issues reported and will keep working hard to make this technology better in each release.

    Thanks,

    Juan

    Comment


      #17
      Thanks for your reply here Juan!

      All I need to be happy for now is raytraced shadows and raytraced AO, working with all engine features (including foliage and VR) with good performance. Reflections and GI are heavy, and in my opinion it wouldn't be bad if you guys at Epic would focus most of the work on getting all engine features to work perfectly with RT shadows and AO, because those are the things that are quite close to performance of CSM and SSAO, while being vastly superior in quality. I already spend like 35% of my games frame budget on CSM, moving that to RT shadows is something that might even be faster in some cases.

      I'm obviously excited about things like reflections and GI too, but I don't anticipate shipping a game with those in the near future. Shadows and AO is in theory very much usable now already, once things like HISMC performance and VR support have been worked on in the engine. I hope (and I'm optimistic) that 4.24 will be an engine version where replacing CSM with RT shadows (on Turing GPUs) will make sense both from a quality and performance perspective in a foliage-heavy VR game.
      Last edited by John Alcatraz; 09-02-2019, 02:00 AM.
      Easy to use UMG Mini Map on the UE4 Marketplace.
      Forum thread: https://forums.unrealengine.com/show...-Plug-and-Play

      Comment


        #18
        Originally posted by Juan Canada View Post
        Hi,

        We have been reviewing most of the issues discussed here, checking if there are regressions, and if we need to take immediate action. So far we have not found any issue introduced in Preview 8 but we are going to keep looking into skylight in reflections and any other topic discussed here to ensure bugs found are fixed as soon as possible.

        There is a second category of issues mentioned here that are not actual bugs but known limitations that need some more time to improve. Namely improving performance with many instances (foliage) or extending denoisers with the ability to support more than one sample per pixel are in this second group. We are working on these areas and will hopefully get good results here in the near future, but these are not things that can be improved in a few days.

        Besides, there are other topics mentioned in this thread that are open problems that do not have a clear solution in real time ray tracing yet. Since DXR compatible hardware was released last year and real time ray tracing techniques started to spread out, I have observed some confusion regarding what people expect from real time ray tracing. Real time ray tracing is not real time path tracing. In real time ray tracing we can trace a very low amount of rays per pixel (5-ish), which allows to solve certain problems pretty successfully (such as area shadows or reflections in cases where screen space reflections do not work) but it cannot solve at 30fps the same problems offline renderers solve tracing thousands of rays per pixel and taking minutes or hours to clean a frame. Results obtained with real time ray tracing are amazing in many cases, but a whole replacement of offline techniques will take time to happen.

        Specifically, problems such seeing ray tracing effects in reflections or refractions are very hard to solve (i,e area shadows in reflections, GI in translucency, etc) These problems cannot be solved in real time yet, we need more research, more novel ideas and eventually faster hardware.

        We will eventually get there. Ray tracing adoption in the film space was quite slow in 2005 because of the limitations of the algorithms at that time, but ten years later it was almost impossible to find a single frame in a movie that was not generated with ray tracing techniques. It will take time until the same happens in games, but some techniques such shadows already show great performance in many scenarios and it looks like this will become the de facto technique for shadows very soon, at least in desktop machines. Other techniques will follow later.

        I also understand some people are not interested in ray tracing yet, and will forward your suggestion of creating a separate subforum. With regards to this topic, do not be concerned about Epic prioritizing ray tracing on top of everything else. There are several major initiatives in the rendering team these days and ray tracing is one of them but there are also others. The rendering development branch is available in Github you so can check the activity in many non ray tracing topics is quite intense (and there are so many cool things in the works ).


        In some cases we might have introduced bugs. That can always happen, specially if the technology is still in beta. In other cases like reflections he issue is more that given the very low ray budget we need to make difficult decisions on when to trace more rays and when to clip or fall back to other mechanism. Ideally we should always provide raytracing passes that when increasing the amount of samples and the quality knows they converge to the output given by the path tracer, but this is not always possible because of a number of reasons (some of them related to performance and others to the shenanigans of splitting the rendering equation into separated ray tracing passes). In any case we are very thankful for the issues reported and will keep working hard to make this technology better in each release.

        Thanks,

        Juan
        I think there may be a bit of a misunderstanding. The only Preview 8 specific issue is that ray traced skylight illumination now does not work *reliably* with static mode, the only mode which provides correct result when skylight has ray tracing enabled. Prior to Preview 8, Static Skylight mobility mode worked more or less always with Ray Tracing. After Preview 8, it sometimes does not work in a fresh project, and randomly starts to work after performing some steps I am unable to reliably determine/reproduce.
        All the other issues mentioned, such incorrect reflection brightness, ray traced skylight illumination being mixed with rasterized skylight diffuse convolution, sky sphere occluding the ray tracing or poor performance are issues that have been around for longer, not just Preview 8.

        I am aware of all the realtime raytracing and DRX limitations, but most of the issues above are not consequence of those limitations, and should be resolved.

        Also, the performance in 4.23 can be gained back by setting r.RayTracing.InstancedStaticMeshes cvar to 0. It should not be *that* hard to automatically detect if all ray tracing effects are disabled, and if so, set r.RayTracing.InstancedStaticMeshes to 0 automatically.

        Aside from the SIM performance above, here's the list of things that are not limitations of DXR or Realtime RayTracing in general and should be fixed in order to make RT more usable in UE4:
        1. When Ray Tracing is enabled on Skylight, Skylight mobility should have no impact on the result. Rasterized diffuse convolution and rasterized reflection convolution of the SkyLight should be disabled, as the skylight illumination is already handled by Ray Tracing.

        2. Denoiser should be able to handle more than 1SPP for reflections. If the denoiser can handle more than 1SPP for all the other ray tracing effects - GI, AO, Area Shadows, there's no reason it should not handle multisampled reflections too.

        3. Ray Traced Skylight illumination should start tracing rays from the distance defined by Sky Distance Threshold value, so that it's automatically prevented from being occluded by skylight background elements, without requiring user to manage their ray tracing visibility flags explicitly. This will also allow them to still be reflected by ray traced reflections, instead of the blurry skylight reflection probe cache. (Think of environment light being MIS sampled area light sphere with inverted normals with the origin being where the Skylight object is and radius of the sphere being the Sky Distance Threshold)
        Last edited by Rawalanche; 09-02-2019, 04:38 AM.
        https://www.artstation.com/artist/rawalanche

        Comment


          #19
          Juan Canada Thank you for the feedback and congratulations for all the team on this already great achievement in so short time. I do follow the work being done in the source code and I know several features are being handled regularly.
          Nilson Lima
          Technical Director @ Rigel Studios Ltda - twitter: @RigelStudios
          Join us at Discord: https://discord.gg/uFFSEXY

          UE4 Marketplace: Cloudscape Seasons
          supporting: Community FREE Ocean plugin

          Comment


            #20
            I feel like GPU ray tracing is in a state similar to programmable shaders when shader model 2.0 came along and people were trying lots of things and still figuring out what worked and what didn't. Like Juan said, current consumer hardware can only trace a couple rays per pixel so novel ways of faking things are needed to make best use of those rays.

            Comment


              #21
              Originally posted by Manoel.Neto View Post
              I feel like GPU ray tracing is in a state similar to programmable shaders when shader model 2.0 came along and people were trying lots of things and still figuring out what worked and what didn't. Like Juan said, current consumer hardware can only trace a couple rays per pixel so novel ways of faking things are needed to make best use of those rays.
              This is not about conserving performance though. This is really just about bugs. The issues mentioned aren't really related to faking things, but rather about ray tracing behaving incorrectly.
              https://www.artstation.com/artist/rawalanche

              Comment


                #22
                Originally posted by Rawalanche View Post

                This is not about conserving performance though. This is really just about bugs. The issues mentioned aren't really related to faking things, but rather about ray tracing behaving incorrectly.
                If that is the case then as some of us suggested, we would all be better off if they have a "beta" section for raytracing dedicated in the forum so we don't have to deal with clutter all day, if some folks want to play around with buggy highly experimental features and report/complain then not my problem just do it in another room so that the rest of us who are trying to make a living out of the engine can pose the valid questions that matters in every day workflow and hope very hardly to get some form of an answer, I say that bluntly. But our hopes of getting any feedback or exposure to our posts can be further diminished by these ever so problematic raytrace posts shooting up our rides. It's becoming like the Steam store here.

                You are trying a technology that may be ready in some limited way in ten years time today, maybe, yes ten years! In ten years I predict you may get only shadows working in a somewhat performant way and not even then all shadows would be raytraced, I put my 2 cent on the table for that prediction just like I did for VR a few years back when that was all the hype!

                Game development has enough tech challenges to screw both ears of any developer last thing we need is another layer of complication proven not working and even if it does work in a hypothetical universe it will be utterly useless in today's markets for the vast majority.
                Other than hard working individuals, our only other chance of any tech feedback are these forums that is all i ask kindly.

                juancanada Thanks for pitching in.

                Comment


                  #23
                  In my project UE4.23 Ray Tracing Reflections completely ignore BSP geometry + it has weird shadows:
                  Attached Files
                  Last edited by shipo; 09-11-2019, 08:00 AM.

                  Comment


                    #24
                    shipo Why do you torture yourself man, why don't you just stick to what works, the good old fashion way.

                    RT is not ready, it will not be ready for many years to come, GPU Rendering in offline renderers only just recently started working somewhat (depending on the project at hand) that is after nearly ten years of pushing hard with no realtime restrictions at all! Can you imagine what it will take to make RT for in game engines actually work on a production level?

                    Of course Huang is going to yell that it works he wants to sell those overpriced cards of his!

                    Comment


                      #25
                      WilliamK I dont know, while I agree with you but I have been quite successful in using raytracing for exteriors/environments. Its not perfect, but it has the things that I have been missing in the last 2 years and I am very happy. Now its not about gaming or such, I just love to use UE4 as a visualization tool too.

                      So I ofcourse agree that the raytracing has many problems, but the technology is still new and discovering the old/new problems that it has are going to help the developers understand the limitations and bugs. So its good thing what shipo is doing discovering problems. I for example didnt knew that BSPs doesnt work with raytraced reflections, now I know.

                      I would like to challenge peoples to understand that UE4 is not only for gaming but it can also be a 3d visualization tool. So whatever doesnt work in gaming, could possibly just work for visualizations aka 2 fps and ansel screenshots/level sequencer rendering.

                      UE4 has very interesting future in virtual production and I cant wait to be involved in such projects in the future.

                      Comment


                        #26
                        I don't want to create another thread, so I just add here.

                        It seems the ray-tracing cast shadows from invisible HLOD proxies.
                        You can see on the roof in the attached pic. Without hlod the shadow is gone.

                        Click image for larger version  Name:	ScreenShot00010.jpg Views:	1 Size:	212.7 KB ID:	1666149

                        edit: btw, I found that HLOD is broken too in 4.23.0, so maybe it's not RT problem after all.
                        Last edited by scha; 09-21-2019, 07:26 AM.
                        Snap Plugin for EditorSnap Plugin for Games
                        Modular Japanese HouseScaffold System
                        TwitterBlogYoutubeitch.io

                        Comment


                          #27
                          Originally posted by William K View Post
                          shipo Why do you torture yourself man, why don't you just stick to what works, the good old fashion way.

                          RT is not ready, it will not be ready for many years to come, GPU Rendering in offline renderers only just recently started working somewhat (depending on the project at hand) that is after nearly ten years of pushing hard with no realtime restrictions at all! Can you imagine what it will take to make RT for in game engines actually work on a production level?

                          Of course Huang is going to yell that it works he wants to sell those overpriced cards of his!
                          Absolutly True.
                          CHECK OUT:- Visit https://3darchstuffs.com/ for Unreal arch-viz tutorials, Unreal Demo EXE and Unreal projects with source files.


                          https://www.youtube.com/playlist?lis...X3Sm2Z9NXEHFV9

                          Comment


                            #28
                            Originally posted by William K View Post
                            shipo Why do you torture yourself man, why don't you just stick to what works, the good old fashion way.

                            RT is not ready, it will not be ready for many years to come, GPU Rendering in offline renderers only just recently started working somewhat (depending on the project at hand) that is after nearly ten years of pushing hard with no realtime restrictions at all! Can you imagine what it will take to make RT for in game engines actually work on a production level?

                            Of course Huang is going to yell that it works he wants to sell those overpriced cards of his!
                            I can easily see RT being close to ready for VFX/Viz purposes, not for games. In fact we're using it already at the moment to produce some high quality visualization. RT reflections combined with GPU lightmass produce results very close to offline rendering but much faster, in some specific cases.

                            "Why don't you stick to what works, the good old fashioned way" is about the worst kind of mindset anyone can have when using any evolving technology. So what are you implying? That the nVidia drops RTX support and goes back to rasterization only GPUs? And that Epic immediately stops development of all the raytracing tech since it's a blind alley? At this point you probably realize how little sense your post makes.

                            Any new software technology has bugs and limitations, and the way to advance the technology is to solve/overcome limitations and fix the bugs. If developers and users adapted the "let's stick to what works" mindset, we would still be using 8086 chips waiting seconds for our 320x200 screens to redraw with 16 color characters.

                            In fact, RT in UE4 is already quite remarkably useful in some cases, if those cases aren't games requiring playable framerates. Thanks to RT development, high end visualization and VFX withing Unreal engine is literally just a few dozen bugfixes away.
                            https://www.artstation.com/artist/rawalanche

                            Comment


                              #29
                              Originally posted by Rawalanche View Post

                              I can easily see RT being close to ready for VFX/Viz purposes, not for games. In fact we're using it already at the moment to produce some high quality visualization. RT reflections combined with GPU lightmass produce results very close to offline rendering but much faster, in some specific cases.

                              "Why don't you stick to what works, the good old fashioned way" is about the worst kind of mindset anyone can have when using any evolving technology. So what are you implying? That the nVidia drops RTX support and goes back to rasterization only GPUs? And that Epic immediately stops development of all the raytracing tech since it's a blind alley? At this point you probably realize how little sense your post makes.

                              Any new software technology has bugs and limitations, and the way to advance the technology is to solve/overcome limitations and fix the bugs. If developers and users adapted the "let's stick to what works" mindset, we would still be using 8086 chips waiting seconds for our 320x200 screens to redraw with 16 color characters.

                              In fact, RT in UE4 is already quite remarkably useful in some cases, if those cases aren't games requiring playable framerates. Thanks to RT development, high end visualization and VFX withing Unreal engine is literally just a few dozen bugfixes away.
                              I get what you are saying and i'm not arguing against the fact to find bugs and move a tech forward, I disagree that UE4 ray tracing is comparable to offline render quality, to my eyes it is very very far off still, and even more time consuming to get somewhat similar results if you do, and if you want to use it for indoor renderings it is even worse. and it will be so even with many bugs fixed in the future. But if it works for your needs then great.

                              As for using UE4 in production I keep saying that it is not only the issue of just rendering inside the engine but also the issue of how a pipeline works with imports/exports/caches/material setups compatible with the engine, massive slowdowns and issues with alembic, lots of crashes with slightly more dense poly scenes and many other related problems with lighting and such.

                              Of course i am speaking of scenes other than just a few static meshes in a arch viz scenario. We concluded 2 projects using a game engine for the first time, client based projects with very tight deadlines allowed us to experiment a little, and it included characters and quality expectations were according to the budget given.
                              But we opted to jump into Unity halfway simply because we couldn't import anything inside UE without the engine massively slowing down with alembic files and sequencer constantly crashing rendering 3K/4k sequences (for antialiasing purposes). We had no time to "optimize our scenes further and unity just handled it fine for what it was.

                              We almost missed our deadlines because of such unexpected problems and even with unity we were wishing to invest in a few more GPU's to go back to rendering offline.

                              The setup times and troubleshooting for such simple scenes is just not worth the rendertime advantages. That is the case with most other teams working in larger vfx houses we know of as well.

                              They would rather invest in a GPU renderfarm which would cost them less than 5K even 2K and have a frame render under a minute with far superior results, less material and light hassles, no import/export issues and times wasted, lots of cheating because you can in a 3d app and lots of compositing layers output for free. fitting right into an existing pipeline.

                              By the time all these are "addressed" in a realtime engine of the future we would get GPU's so fast that offline GPU rendering will be able to render far more complicated scenes with far less or comparable times as in a realtime engine and the gap will narrow down further still.

                              And so unless there is a very special case, investing time to make realitme raytracing in a game enigne work for production requirements is from my personal point of view a waste of time for todays needs as well as for the future.

                              But this doesn't mean not finding bugs and helping it work for games of course. that is entirely different and well welcomed feature. The OP and many others though "seemed to be" trying to make an actual product out of it and I assumed that in any normal given world scenario they would have deadlines and such, which is why I told them to stick what works.

                              If they are just playing around features then so be it.




                              Last edited by William K; 09-22-2019, 09:00 AM.

                              Comment


                                #30
                                Originally posted by William K View Post

                                I get what you are saying and i'm not arguing against the fact to find bugs and move a tech forward, I disagree that UE4 ray tracing is comparable to offline render quality, to my eyes it is very very far off still, and even more time consuming to get somewhat similar results if you do, and if you want to use it for indoor renderings it is even worse. and it will be so even with many bugs fixed in the future. But if it works for your needs then great.

                                As for using UE4 in production I keep saying that it is not only the issue of just rendering inside the engine but also the issue of how a pipeline works with imports/exports/caches/material setups compatible with the engine, massive slowdowns and issues with alembic, lots of crashes with slightly more dense poly scenes and many other related problems with lighting and such.

                                Of course i am speaking of scenes other than just a few static meshes in a arch viz scenario. We concluded 2 projects using a game engine for the first time, client based projects with very tight deadlines allowed us to experiment a little, and it included characters and quality expectations were according to the budget given.
                                But we opted to jump into Unity halfway simply because we couldn't import anything inside UE without the engine massively slowing down with alembic files and sequencer constantly crashing rendering 3K/4k sequences (for antialiasing purposes). We had no time to "optimize our scenes further and unity just handled it fine for what it was.

                                We almost missed our deadlines because of such unexpected problems and even with unity we were wishing to invest in a few more GPU's to go back to rendering offline.

                                The setup times and troubleshooting for such simple scenes is just not worth the rendertime advantages. That is the case with most other teams working in larger vfx houses we know of as well.

                                They would rather invest in a GPU renderfarm which would cost them less than 5K even 2K and have a frame render under a minute with far superior results, less material and light hassles, no import/export issues and times wasted, lots of cheating because you can in a 3d app and lots of compositing layers output for free. fitting right into an existing pipeline.

                                By the time all these are "addressed" in a realtime engine of the future we would get GPU's so fast that offline GPU rendering will be able to render far more complicated scenes with far less or comparable times as in a realtime engine and the gap will narrow down further still.

                                And so unless there is a very special case, investing time to make realitme raytracing in a game enigne work for production requirements is from my personal point of view a waste of time for todays needs as well as for the future.

                                But this doesn't mean not finding bugs and helping it work for games of course. that is entirely different and well welcomed feature. The OP and many others though "seemed to be" trying to make an actual product out of it and I assumed that in any normal given world scenario they would have deadlines and such, which is why I told them to stick what works.

                                If they are just playing around features then so be it.



                                You were not reading my post carefully. I was saying we've come close to offline renderer quality by combining ray traced reflections with GPU lightmass. The lighting was mostly prebaked by GPU lightmass, so the diffuse indirect illumination was quite accurate. Previously, that would still leave you with inaccurate specular indirect illumination, since neither reflection captures nor SSR would do the trick, but GPU lightmass combined with RT reflections really looked quite close to offline render quality.

                                We did not fully raytrace the scene. It was not like Vray or Corona or something, but just using the reflection component of the Ray Tracing itself already proved to be useful for a practical project, even at the current, experimental stage. That was my point

                                I was also only talking about Ray Tracing specifically, which is the topic of this thread. Yes, UE4 has a large amount of issues which generally make it questionable for use non-gaming industries, many of which I've broken my teeth on, but introduction of Ray Tracing and continuous development of it is actually a direction of removing at least some of those issues. All in all, it's a good thing.

                                And I am the OP by the way. By the label of the thread itself you can clearly tell I am also a bit critical of the current state of Ray Tracing in UE4, but I'm not abandoning the hope. Epic is not Autodesk after all
                                Last edited by Rawalanche; 09-22-2019, 10:18 AM.
                                https://www.artstation.com/artist/rawalanche

                                Comment

                                Working...
                                X