Ray Tracing is beyond broken in 4.23

Hi,

I am creating this thread to post all the reasons Ray Tracing is broken in 4.23. I’ve already reported quite a few of them in the bug submission form, but I doubt all of the fixes will get the priority, so hopefully creating some waves here on the forum will pull some string.

Let’s start with impossibility to create proper image based lighting in 4.23:

Skylight has 3 mobility modes: Static, Stationary and Movable. All produce very different results with Ray Traced skylight enabled. Up until 4.23 Preview 7, Static mode worked and was only mode that produced correct results. Since Preview 8, Static mode no longer works, and both Stationary and Movable modes produce extremely wrong results.

First, let’s render ourselves a path traced reference, which looks more or less correct:


Now, this is ray traced skylight in Static mode. This mode has worked up until Preview 7 and produced relatively similar results. In Preview 8 it now no longer works anymore:
Now let’s switch to stationary mode. You can see that while some tracing of the skylight environment lighting is happening, reflection seem to reflect unshaded scene for secondary bounces. This severely compromises accuracy:
I logically assumed it would be consequence of “max roughness” threshold for raytracing, where rays above certain roughness reflect rasterized scene reflection probes instead of ray traced reflections. So I went to ray traced reflection settings, and adjusted all the settings for maximum accuracy: Max. Roughness all the way to 1, 16 bounces, 4 samples, and level of shadow reflections all the way up to area shadows. Instead of more accurate results, I got even LESS accurate result. It appears that the rasterized reflection convolution remains added on top:
I proceeded to revert the reflection settings back to their defaults, and switched SkyLight mobility to movable. As you can see this also adds diffuse environment light convolution on top of the raytraced environment light, completely ruining any purpose of the raytraced environment:

This poses many issues:

  1. Skylight mobility should never ever affect shading when ray traced skylight is enabled
  2. Any rasterized reflection/diffuse convolution effects should not be mixed with raytraced skylight.
  3. Most of the game engine users/developers don’t have nearly enough experience with offline rendering to know if they are getting the right result or not. Due to this reason, it’s all the more dangerous that they are getting severely inaccurate results out of the box.

EDIT: So apparently Static Skylight mode still works with Ray Tracing in 4.23 P8 but under a random circumstances I am unable to reproduce. This is a result with Static Skylight and enabled GI:

You can see that static mode more or less matches up with Path Tracer reference. The issue here is that out of the box, in a new project, Static Skylight does not work with Ray Tracing and the steps to make it work are not clear.

Next one:

By default, this is the kind of result new user will get when trying to use Ray Traced SkyLight with the usual unreal environment light setup:


It is not obvious for many that in order for Ray Traced skylight to works, the environment sphere captured by SkyLight capture needs to have Ray Tracing disabled:

The problem is that when Ray Tracing of the Environment sphere is disabled, Ray Tracing instead reflects the low resolution Skylight Capture sphere. This forces user to make very uncomfortable choice between accurate environment reflections or functional environment skylight illumination:

Next one:

Many users are probably not aware that setting Ray Traced Reflections samples to above 1 disables Ray Traced Reflections denoiser. What this effectively means that increasing RTR sample count can in many cases result in both worse performance as well as worse quality.

Here’s an example with 1 samples. You can see denoiser doing its job:

Here’s the very same scene and angle with 2 RTR samples. You can see denoiser stopped working. This is all the more trap to fall into considering this denoiser behavior is unique to reflections only:

Here we go again:

In 4.22 it was possible to have Ray Tracing enabled in project and have no performance hit as long as you do not use any Ray Traced effects:


That is no possible in 4.23. As long as you have RayTracing enabled and use lots of foliage, GenerateVisibleRayTracingMeshCommands will kill the performance, even if ALL the RT effects are OFF:

Hi Rawalanche,

We are sorry to hear about the issues you are experiencing. Ray tracing is still tagged as an “Early Access” technology, while we are happy with what we have done so far, this is just the beginning and we are working hard to make it better every day.

I have already filed some of the things you report here:

  • When ray tracing effects are disabled (r.RayTracing.ForceAllRayTracingEffects is set to 0), there is still some ray tracing work done. In fact it has been like that from the beginning but now that in 4.23 we support heavier geometry types such landscapes or HISM this is more evident. This will be improved in the near future, hopefully in the first 4.23 hotfix

  • “It is not obvious for many that in order for Ray Traced skylight to works, the environment sphere captured by SkyLight capture needs to have Ray Tracing disabled”
    We are aware of this issue and are thinking on strategies for improve it. We have set the “visible in ray tracing” flag to false for the default sky primitives included with the engine but nothing prevents users from use other sky spheres. We are discussing the best way to go here and will hopefully come up with a better solution than the current one soon.

  • We will also deeply look into the issues you report with reflections. I have double checked there are no changes in Preview 8 that should affect this but anyway we will review what we are doing and eventually will improve and fix whatever issues we find, trying to get the best performance vs quality balance.

Best,

   Juan

Since you have the path tracer implementation, it should not be hard to test and verify if ray tracing components are providing correct results. I am very surprised so many weird quirks have gotten through under supervision of someone who has already done one commercial offline renderer, especially one where accuracy was the main priority :wink:

BTW, I am confused when you said “Default sky primitives”. I had no idea the engine had any. I did not find a single sphere in the engine content that had normals properly flipped inside.

It works quite well even on my non RTX 1080Ti card. I am actually very excited about it, as I use UE4 to do also VFX, not just games, so I don’t need 60+FPS performance. That being said, I’d like the accuracy to improve, that’s why I crated this thread. Right now, there’s too many odd things about the raytracing which compromise quality, which makes me feel quite unsure when using it.

i agree with the previous poster, even for vfx you will end up wasting time troubleshooting and missing deadlines on any serious project while youbcan render the scene in vray or arnold or even cycles with infinitely superior results.

​​​​​​edit: i would like to add that it really ****** me off to have countless and endless threads and posts talk about a very niche extremely exprerimental feature like raytracing when in fact there are a hundred other far more important subject matters still behind in the engine that matters for every day production and that the devs could use the attention pointed at by the vast community. Instead everywhere i look we get ten raytracing threads for one valid question.

With no disrespect sometimes i feel like people act like little children who forget why they were crying about trying to accomplish a task when daddy distracts them with a big candy bar! I remember a similar thing happened when epic introduced editing in vr inside the viewport supposedly the future of game dev thats right next is wiring your brain and inserting yourself into the epic matrix.

edit2: can i suggest to epic to create a section in the forums just for raytracing please! Im tired of seeing legit posts end up at the bottom of the page in a day becuae they are choked by ten raytrace related posts.

I have been outside discussions about realtime raytracing mainly because time is precious and I have been forced to look into it mainly because people who use my developed assets asks if it will work or not, so reading this thread makes me even more worry, but there are points I would like to state on my observations and concerns:

Facts on what my usage of the engine are:

  • I have an asset in the marketplace Cloudscape Seasons which is a sky system using Raymarching in material with Alpha-Composite double sided and the cube mesh I apply the material covers the entire scene, so the clouds are rendered inside;
  • I have an ocean being developed using FFT, with realism in mind, translucent material with tessellation applied in a quad-tree set of planes;
  • I have another sky system using voxels, which is also heading towards realism and performance;

Facts about current implementation for realtime raytracing as in 4.23 preview 8:

  • does not render reflections of translucent materials, so if you apply a translucent material in a sphere and place it in front of a mirror you won’t see the sphere reflected;
  • it changes the material appearance of alpha-composite (as shown in the video) and somehow disregard the blending mode showing edges (this should be ok for translucency, but not alpha composite), is using refraction (not sure, but check the video) and generating spherical artifacts;
  • the exponential height fog placed in the scene and set to volumetric fog is shown in the mirror reflection, but changes on color are not being reflected on the mirror image, and I can’t see the difference when volumetric fog is disable and re-enabled, which is odd, the fog is removed if I make it invisible or remove from the scene, so at least part of the reflection is there, which gives me hope.
  • translucent materials like water, what is necessary for it to work?

Considerations:

  • while in Beta, I understand the feature can be partial in several things;
  • it is hard to use it as it is, and I am very pleased to see that all the demos shown, up to now, could show it working and not being dependent on the problematic features, but real case scenarios will be full of dependency on the features not ready yet, which will give a huge frustration for everyone putting their hands on it right now;

I have a film, which would be great to have realtime raytracing working, not even concerns me the realtime part since I will use shots from each frame, but the features missing completion is a major concern. It would be great to have a switch at material instance level to allow the material instance to behave as in raster mode, but I know this is a pain. I really hope the feature, when out of Beta ahead, will work properly in every scenario.

The video where I show what I am talking about here:

watch?v=BtYWxyPI-iY

I agree, maybe a section for beta features and inside sections for each beta feature.

Independent from the question if disabling raytracing will make the cost go away, having GenerateVisibleRayTracingMeshCommands take 18 ms of time for a bunch of grass does not sound too ideal, as that means it’s quite impossible to use it with foliage, right? Is this just an early implementation that can be optimized, or is there some deeper problem with foliage that makes it very slow to process for RT?

Please also have a look at the foliage lighting model cause SSS is still completely fubar. I know that’s no biggy cause who needs to render trees or foliage anyways but for those of us that fancy doing things with green stuff t rly would be much appreciated (yes it’s difficult cause Reeeeee CSM depth for SSS easier than experimental RT Wooooo but you guys can do this, you rly can! If you can make nice Gold Crown in water reflections this should be a breeze!)

Hi all,

I’m trying to read all the relevant posts on the this subject. As you can see from my Youtube channel,(user/tetanoisee) I’ve been doing ue4 / raytrace tests since the dev rendering stage, and “rawalance” summarizes all the problems I’ve encountered.

in my opinion, the current consumer GPUs are not enough to use all the features of raytrace for "real time/ 60fps ". I don’t think this is related to the software side of the problem. I think it’s all about hardware power. If you look at the videos on my YouTube channel, even at simple levels, all raytrace features are enabled and sample values ​​are increased, even rtx2080ti suffers from fps)

Of course, it is not surprising that we use the game engine as the primary point of view in terms of performance/quality. however, existing GPUs seem inadequate. (perhaps the algorithms can be strengthened as the screenspace for real-time performance, but this time conflicts with the purpose of use because the accuracy of raytrace is lost)

like other users, I tried UE4 in different usage scenarios. I focused more on the export sequencer animation. All the features in my 4k 60 fps export attempt still didn’t work. (usually with high sample values, crash or noisy export)

in the light of all these results;

  • Existing consumer GPUs are insufficient in the “Real Time” section for this Raytrace/ dxr technology. (I’m not just talking about reflection; I’m talking about performance when using all the gi, ao, reflection, refraction, and shadow features on the raytrace.)

We should be able to use the export sequencer effectively for raytrace. I think this section is hardware independent. hopefully version 4.23 can improve this section (crash and noise problem in high sample)

-One of the biggest problems I’ve seen in my tests is the shadow part of translucent objects (I’m not talking about translucent shadow, I’m talking about whether shadow is active or passive.huge difference on interior lighting.)

-After many tests, I can easily say that there are differences between 4.22 and 4.23 about RTGI, lights, foliage lighting and denoiser.

same level migrated from 4.22.3 to 4.23 P8 here is results ;

(low rtgi samples)
with denoiser :ue4 (4.22.3 vs 4.23 P8) dxr compare - Imgsli

without denoiser : ue4 (4.22.3 vs 4.23 P8) dxr compare - Imgsli

I don’t know what’s the difference from the technical point of view but;
i think …
-rtgi works more accurately at 4.22.3
-denoiser works better at 4.23 p8
-I don’t know if -foliage was supported in 4.22.3?
-Light falloff values are not the same. (Is it normal to have so much difference that I migrate to the same level?)
-the two-sided processing of some objects has changed.

In the forum, I would like to discuss raytrace as own section … where do I sign? :smiley:

I would like to congratulate the whole team, especially the juan canada. I’ve been doing 3D visualization for 20 years and it’s really exciting to experience the point where technology comes from.

I’m sorry about my bad English.

Hi,

We have been reviewing most of the issues discussed here, checking if there are regressions, and if we need to take immediate action. So far we have not found any issue introduced in Preview 8 but we are going to keep looking into skylight in reflections and any other topic discussed here to ensure bugs found are fixed as soon as possible.

There is a second category of issues mentioned here that are not actual bugs but known limitations that need some more time to improve. Namely improving performance with many instances (foliage) or extending denoisers with the ability to support more than one sample per pixel are in this second group. We are working on these areas and will hopefully get good results here in the near future, but these are not things that can be improved in a few days.

Besides, there are other topics mentioned in this thread that are open problems that do not have a clear solution in real time ray tracing yet. Since DXR compatible hardware was released last year and real time ray tracing techniques started to spread out, I have observed some confusion regarding what people expect from real time ray tracing. Real time ray tracing is not real time path tracing. In real time ray tracing we can trace a very low amount of rays per pixel (5-ish), which allows to solve certain problems pretty successfully (such as area shadows or reflections in cases where screen space reflections do not work) but it cannot solve at 30fps the same problems offline renderers solve tracing thousands of rays per pixel and taking minutes or hours to clean a frame. Results obtained with real time ray tracing are amazing in many cases, but a whole replacement of offline techniques will take time to happen.

Specifically, problems such seeing ray tracing effects in reflections or refractions are very hard to solve (i,e area shadows in reflections, GI in translucency, etc) These problems cannot be solved in real time yet, we need more research, more novel ideas and eventually faster hardware.

We will eventually get there. Ray tracing adoption in the film space was quite slow in 2005 because of the limitations of the algorithms at that time, but ten years later it was almost impossible to find a single frame in a movie that was not generated with ray tracing techniques. It will take time until the same happens in games, but some techniques such shadows already show great performance in many scenarios and it looks like this will become the de facto technique for shadows very soon, at least in desktop machines. Other techniques will follow later.

I also understand some people are not interested in ray tracing yet, and will forward your suggestion of creating a separate subforum. With regards to this topic, do not be concerned about Epic prioritizing ray tracing on top of everything else. There are several major initiatives in the rendering team these days and ray tracing is one of them but there are also others. The rendering development branch is available in Github you so can check the activity in many non ray tracing topics is quite intense (and there are so many cool things in the works :slight_smile: ).

In some cases we might have introduced bugs. That can always happen, specially if the technology is still in beta. In other cases like reflections he issue is more that given the very low ray budget we need to make difficult decisions on when to trace more rays and when to clip or fall back to other mechanism. Ideally we should always provide raytracing passes that when increasing the amount of samples and the quality knows they converge to the output given by the path tracer, but this is not always possible because of a number of reasons (some of them related to performance and others to the shenanigans of splitting the rendering equation into separated ray tracing passes). In any case we are very thankful for the issues reported and will keep working hard to make this technology better in each release.

Thanks,

   Juan

Thanks for your reply here Juan!

All I need to be happy for now is raytraced shadows and raytraced AO, working with all engine features (including foliage and VR) with good performance. Reflections and GI are heavy, and in my opinion it wouldn’t be bad if you guys at Epic would focus most of the work on getting all engine features to work perfectly with RT shadows and AO, because those are the things that are quite close to performance of CSM and SSAO, while being vastly superior in quality. I already spend like 35% of my games frame budget on CSM, moving that to RT shadows is something that might even be faster in some cases.

I’m obviously excited about things like reflections and GI too, but I don’t anticipate shipping a game with those in the near future. Shadows and AO is in theory very much usable now already, once things like HISMC performance and VR support have been worked on in the engine. I hope (and I’m optimistic) that 4.24 will be an engine version where replacing CSM with RT shadows (on Turing GPUs) will make sense both from a quality and performance perspective in a foliage-heavy VR game.

I think there may be a bit of a misunderstanding. The only Preview 8 specific issue is that ray traced skylight illumination now does not work reliably with static mode, the only mode which provides correct result when skylight has ray tracing enabled. Prior to Preview 8, Static Skylight mobility mode worked more or less always with Ray Tracing. After Preview 8, it sometimes does not work in a fresh project, and randomly starts to work after performing some steps I am unable to reliably determine/reproduce.
All the other issues mentioned, such incorrect reflection brightness, ray traced skylight illumination being mixed with rasterized skylight diffuse convolution, sky sphere occluding the ray tracing or poor performance are issues that have been around for longer, not just Preview 8.

I am aware of all the realtime raytracing and DRX limitations, but most of the issues above are not consequence of those limitations, and should be resolved.

Also, the performance in 4.23 can be gained back by setting r.RayTracing.InstancedStaticMeshes cvar to 0. It should not be that hard to automatically detect if all ray tracing effects are disabled, and if so, set r.RayTracing.InstancedStaticMeshes to 0 automatically.

Aside from the SIM performance above, here’s the list of things that are not limitations of DXR or Realtime RayTracing in general and should be fixed in order to make RT more usable in UE4:

  1. When Ray Tracing is enabled on Skylight, Skylight mobility should have no impact on the result. Rasterized diffuse convolution and rasterized reflection convolution of the SkyLight should be disabled, as the skylight illumination is already handled by Ray Tracing.

  2. Denoiser should be able to handle more than 1SPP for reflections. If the denoiser can handle more than 1SPP for all the other ray tracing effects - GI, AO, Area Shadows, there’s no reason it should not handle multisampled reflections too.

  3. Ray Traced Skylight illumination should start tracing rays from the distance defined by Sky Distance Threshold value, so that it’s automatically prevented from being occluded by skylight background elements, without requiring user to manage their ray tracing visibility flags explicitly. This will also allow them to still be reflected by ray traced reflections, instead of the blurry skylight reflection probe cache. (Think of environment light being MIS sampled area light sphere with inverted normals with the origin being where the Skylight object is and radius of the sphere being the Sky Distance Threshold)

[USER=“1224082”]Juan Canada[/USER] Thank you for the feedback and congratulations for all the team on this already great achievement in so short time. I do follow the work being done in the source code and I know several features are being handled regularly.

I feel like GPU ray tracing is in a state similar to programmable shaders when shader model 2.0 came along and people were trying lots of things and still figuring out what worked and what didn’t. Like Juan said, current consumer hardware can only trace a couple rays per pixel so novel ways of faking things are needed to make best use of those rays.

This is not about conserving performance though. This is really just about bugs. The issues mentioned aren’t really related to faking things, but rather about ray tracing behaving incorrectly.

If that is the case then as some of us suggested, we would all be better off if they have a “beta” section for raytracing dedicated in the forum so we don’t have to deal with clutter all day, if some folks want to play around with buggy highly experimental features and report/complain then not my problem just do it in another room so that the rest of us who are trying to make a living out of the engine can pose the valid questions that matters in every day workflow and hope very hardly to get some form of an answer, I say that bluntly. But our hopes of getting any feedback or exposure to our posts can be further diminished by these ever so problematic raytrace posts shooting up our rides. It’s becoming like the Steam store here.

You are trying a technology that may be ready in some limited way in ten years time today, maybe, yes ten years! In ten years I predict you may get only shadows working in a somewhat performant way and not even then all shadows would be raytraced, I put my 2 cent on the table for that prediction just like I did for VR a few years back when that was all the hype!

Game development has enough tech challenges to screw both ears of any developer last thing we need is another layer of complication proven not working and even if it does work in a hypothetical universe it will be utterly useless in today’s markets for the vast majority.
Other than hard working individuals, our only other chance of any tech feedback are these forums that is all i ask kindly.

@juancanada Thanks for pitching in.