Ray Tracing is beyond broken in 4.23

[Rawalanche]

@Juan_Canada

Hi,

We have been reviewing most of the issues discussed here, checking if there are regressions, and if we need to take immediate action. So far we have not found any issue introduced in Preview 8 but we are going to keep looking into skylight in reflections and any other topic discussed here to ensure bugs found are fixed as soon as possible.

There is a second category of issues mentioned here that are not actual bugs but known limitations that need some more time to improve. Namely improving performance with many instances (foliage) or extending denoisers with the ability to support more than one sample per pixel are in this second group. We are working on these areas and will hopefully get good results here in the near future, but these are not things that can be improved in a few days.

Besides, there are other topics mentioned in this thread that are open problems that do not have a clear solution in real time ray tracing yet. Since DXR compatible hardware was released last year and real time ray tracing techniques started to spread out, I have observed some confusion regarding what people expect from real time ray tracing. Real time ray tracing is not real time path tracing. In real time ray tracing we can trace a very low amount of rays per pixel (5-ish), which allows to solve certain problems pretty successfully (such as area shadows or reflections in cases where screen space reflections do not work) but it cannot solve at 30fps the same problems offline renderers solve tracing thousands of rays per pixel and taking minutes or hours to clean a frame. Results obtained with real time ray tracing are amazing in many cases, but a whole replacement of offline techniques will take time to happen.

Specifically, problems such seeing ray tracing effects in reflections or refractions are very hard to solve (i,e area shadows in reflections, GI in translucency, etc) These problems cannot be solved in real time yet, we need more research, more novel ideas and eventually faster hardware.

We will eventually get there. Ray tracing adoption in the film space was quite slow in 2005 because of the limitations of the algorithms at that time, but ten years later it was almost impossible to find a single frame in a movie that was not generated with ray tracing techniques. It will take time until the same happens in games, but some techniques such shadows already show great performance in many scenarios and it looks like this will become the de facto technique for shadows very soon, at least in desktop machines. Other techniques will follow later.

I also understand some people are not interested in ray tracing yet, and will forward your suggestion of creating a separate subforum. With regards to this topic, do not be concerned about Epic prioritizing ray tracing on top of everything else. There are several major initiatives in the rendering team these days and ray tracing is one of them but there are also others. The rendering development branch is available in Github you so can check the activity in many non ray tracing topics is quite intense (and there are so many cool things in the works ).

In some cases we might have introduced bugs. That can always happen, specially if the technology is still in beta. In other cases like reflections he issue is more that given the very low ray budget we need to make difficult decisions on when to trace more rays and when to clip or fall back to other mechanism. Ideally we should always provide raytracing passes that when increasing the amount of samples and the quality knows they converge to the output given by the path tracer, but this is not always possible because of a number of reasons (some of them related to performance and others to the shenanigans of splitting the rendering equation into separated ray tracing passes). In any case we are very thankful for the issues reported and will keep working hard to make this technology better in each release.

Thanks,

Juan

I think there may be a bit of a misunderstanding. The only Preview 8 specific issue is that ray traced skylight illumination now does not work reliably with static mode, the only mode which provides correct result when skylight has ray tracing enabled. Prior to Preview 8, Static Skylight mobility mode worked more or less always with Ray Tracing. After Preview 8, it sometimes does not work in a fresh project, and randomly starts to work after performing some steps I am unable to reliably determine/reproduce.
All the other issues mentioned, such incorrect reflection brightness, ray traced skylight illumination being mixed with rasterized skylight diffuse convolution, sky sphere occluding the ray tracing or poor performance are issues that have been around for longer, not just Preview 8.

I am aware of all the realtime raytracing and DRX limitations, but most of the issues above are not consequence of those limitations, and should be resolved.

Also, the performance in 4.23 can be gained back by setting r.RayTracing.InstancedStaticMeshes cvar to 0. It should not be that hard to automatically detect if all ray tracing effects are disabled, and if so, set r.RayTracing.InstancedStaticMeshes to 0 automatically.

Aside from the SIM performance above, here’s the list of things that are not limitations of DXR or Realtime RayTracing in general and should be fixed in order to make RT more usable in UE4:

1. When Ray Tracing is enabled on Skylight, Skylight mobility should have no impact on the result. Rasterized diffuse convolution and rasterized reflection convolution of the SkyLight should be disabled, as the skylight illumination is already handled by Ray Tracing.

2. Denoiser should be able to handle more than 1SPP for reflections. If the denoiser can handle more than 1SPP for all the other ray tracing effects - GI, AO, Area Shadows, there’s no reason it should not handle multisampled reflections too.

3. Ray Traced Skylight illumination should start tracing rays from the distance defined by Sky Distance Threshold value, so that it’s automatically prevented from being occluded by skylight background elements, without requiring user to manage their ray tracing visibility flags explicitly. This will also allow them to still be reflected by ray traced reflections, instead of the blurry skylight reflection probe cache. (Think of environment light being MIS sampled area light sphere with inverted normals with the origin being where the Skylight object is and radius of the sphere being the Sky Distance Threshold)