Come and see for yourself: Volume materials takes ages to compile

Development Rendering
, ,
1

I know Unreal’s meme about compiling shaders, but it’s only when you open a brand new project.

Here I give you one simple material that’s going to freeze the editor for a while.

On my side, if I try to save this material on UE 5.0, my i7-10700K will be “Finishing Compiling Shaders” for more than 10 minutes, and the shader compile workers will exhaust my 32GB of RAM, resorting to swapping.

Here it is:

image
image1223×543 105 KB

and the pastebin:

The worse part is that it seems not to be specific to my graph, but general to the volume material domain.
Having tried more complex setups with a couple texture samples and some maths functions, the compile time of the one material ballooned to take about 10 hours. During that time, my machine peaked in total at about 100GB of committed memory.

Could someone verify that it’s not just me with that problem ? That someone else somewhere can see that it takes unreasonably long to compile volume materials ?

PS: If you test, leave all usage flags unchecked. Just use the default material settings with Volume domain, and Additive blending.

PPS: Oh and of course, if anyone’s got a solution to lower the compile time to something decent, I’ll take it xD

2 Likes
2

I just tried with my computer at work (an AMD Threadripper 3960x - 24 cores and 48 Threads, 64GB Ram), and it took about 9mins to compile on the launcher’s UE 5.0.1.

Here’s the stats

LogShaderCompilers: Display: === Shader Compilation stats ===
LogShaderCompilers: Display: Shaders Compiled: 15449
LogShaderCompilers: Display: Jobs assigned 15416, completed 15416 (100.00%)
LogShaderCompilers: Display: Average time worker was idle: 0.54 s
LogShaderCompilers: Display: Time job spent in pending queue: average 243.43 s, longest 656.91 s
LogShaderCompilers: Display: Job execution time: average 15.82 s, max 511.67 s
LogShaderCompilers: Display: Job life time (pending + execution): average 259.25 s, max 659.84
LogShaderCompilers: Display: Time at least one job was in flight (either pending or executed): 1551.07 s
LogShaderCompilers: Display: Jobs were issued in 1767 batches (only local compilation was used), average 8.72 jobs/batch
LogShaderCompilers: Display: Average processing rate: 9.94 jobs/sec
LogShaderCompilers: Display: Effective parallelization: 18.49 (times faster than compiling all shaders on one thread). Compare with number of workers: 24
LogShaderCompilers: Display: Top 5 most expensive shader types by average time:
LogShaderCompilers: Display:                           FRenderVolumetricCloudRenderViewCS (compiled   34 times, average 163.33 sec, max 511.52 sec, min 10.47 sec)
LogShaderCompilers: Display:                                          FClusteredShadingPS (compiled    4 times, average 107.31 sec, max 227.83 sec, min 4.88 sec)
LogShaderCompilers: Display:                                   FCloudRenderViewPS111false (compiled    2 times, average 83.39 sec, max 145.44 sec, min 21.34 sec)
LogShaderCompilers: Display:                                   FCloudRenderViewPS011false (compiled    2 times, average 83.39 sec, max 145.65 sec, min 21.13 sec)
LogShaderCompilers: Display:                                   FCloudRenderViewPS110false (compiled    2 times, average 80.05 sec, max 145.19 sec, min 14.91 sec)
LogShaderCompilers: Display: Top 5 shader types by total compile time:
LogShaderCompilers: Display:                           FRenderVolumetricCloudRenderViewCS - 19.36% of total time (compiled   34 times, average 163.33 sec, max 511.52 sec, min 10.47 sec)
LogShaderCompilers: Display:                                 TBasePassVSFNoLightMapPolicy - 3.28% of total time (compiled  431 times, average 2.18 sec, max 4.05 sec, min 0.65 sec)
LogShaderCompilers: Display:                                             FDebugViewModeVS - 3.18% of total time (compiled  428 times, average 2.13 sec, max 4.10 sec, min 0.64 sec)
LogShaderCompilers: Display:                                 TBasePassPSFNoLightMapPolicy - 2.91% of total time (compiled  443 times, average 1.88 sec, max 5.72 sec, min 0.73 sec)
LogShaderCompilers: Display:                                                 FLumenCardVS - 2.84% of total time (compiled  429 times, average 1.90 sec, max 3.58 sec, min 0.29 sec)
LogShaderCompilers: Display: ================================================

Also, I narrowed down that the noise node is the culprit. Without the noise node, the compile times are normal again.

Is this the point where I should submit a bug to the tracker ?

3 Likes
3

You can, but for what it is worth I reported this a year(?) ago and was never given a tracking number so it may just be by design.

In general you should be using a volume texture, not the noise node, but if you are insistent on using it then one way I found to speed up the compilation process is to disable raymarched shadows in the advanced volumetric output while you lookdev your material. I’m guessing the raymarching loops don’t like the noise function. It will still take a while to compile, but it should be significantly faster.

Once you’ve finished your material you can turn raymarched shadows back on for the final build.

It’s kind of disappointing, because cell noise is actually faster than a texture lookup in some cases.

2 Likes
5

I’m experiencing the exact same issue, with a very similar material. It’s like the most basic material you can create for doing local volumetrics that have noise. Even with a noise texture sample, it still takes like a minute or so to compile. With a noise node, it’s taking like 10 minutes to compile.

This is on UE5.0.3

1 Like
6

omg thank you so much!

1 Like