how you do that?
(sorry it’s in german) There is a plugin on the marketplace which does this prioritizing automatically, but did not tried it. Doing the priority on all ShaderCompileWorkers usually gives them like a 1/3 more CPU usage and are more consistent…
Anyway, this is a new plugin seeming to tackle this topic. Also not tried, but maybe something to mention (and its free):
will try this, thank you, the shader compile time is not that long but if windows decides is not important and will throttle the execution that is not fun
I already finished this project. In order to render alpha channel in the sequencer render queue, I had to go to the project settings and turn on “enable alpha”. The editor reminded me to restart itself. So I did, to my surprise, when the splash screen showed up, it recompiled 8000+ shaders again. 2 hours later, the editor was fully loaded. Then it started “preparing shaders (8,000+)”. Basically another day is wasted.
Who would have thought that this one tiny setting would waste so much time?
Btw, I have the shader optimization tool. But it doesn’t give me an opportunity to stop these 2 super slow processes.
How should I work with it like this?
Uou are still compiling +8000 shaders?
1 Like
The very first time it has to compile all of the shaders inside the engines and plugin
If you let it complete then it should be faster every time
If you don’t have a lot of space in the drive with the ddc it will only use a certain amount then start deleting the oldest for new compiles
so I put the ddc on its own
currently my ddc is about 60 G
when you load a level and it compiles shaders you have to save it, so it saves the shader build data. if you close without saving those shader compiles go poof
I think projects on the marketplace should have precompiled shaders, but that could quadruple filesizes
There is also the ddc commandline shader compile
UE\Engine\Binaries\Win64\UE4Editor.exe ProjectName -run=DerivedDataCache -fill
Thank you, what does this command do? Does it force UE5 to compile? How would it save time?
UE\Engine\Binaries\Win64\UE4Editor.exe ProjectName -run=DerivedDataCache -fill
Thanks again.
Yeah, I realized that these things will likely trigger UE to recompile 8000+ shaders, TWICE, every time:
1, installing some certain plugins.
2, making some changes in the project settings, such as “enable alpha channel in editor → rendering” and many other settings. (feel free to insert your discovery here.)
3, not very likely, but sometimes, adding some marketplace assets into your existing project.
4, updating either UE or plugin.
5, unknown reasons.
What’s your experience? Why is this issue not bothering majority of users, but only us?
when first time I run engine it compiles shaders and programs and plugins only one time only one time and this is all. I have not experienced this issue , maybe the reason is I am using ue .20
you run it at night
that’s what epic does each night
they run a batch file with that commandset with the names and relative paths of each project and it compiles the shaders overnight and fills the ddc
now
to reduce the number of shaders compiling for your project
turn off any platforms you are not building for.
If you are making a quest android project
don’t have any other platform checked
or it will compile shaders for all them too
Don’t have unnecessary asset packs and materials
all the matierials will compile even if you aren’t using them
so
you have the sample materials installed, but all you are using is 3 mats
move them and delete the rest
uncheck ogl or vulkan, only choose 1 or it will compile a separate shader for each
8000 shaders is normal
city park environment is 13000, factory environment is 9500
1 Like
UE5 added:
On Demand Shader Compilation (ODSC) - Only compile shaders that are used to draw instead of compiling every shader in each Material’s shader map on PostLoad. This is an experimental feature that is still being developed. This change ensures the editor only compiles shaders it needs to render, resulting in approximately 60% less shaders being compiled and faster start-up times for the editor when opening maps.
DefaultEngine.ini:
[SystemSettings]
r.ShaderCompiler.JobCacheDDC=1
5 Likes
I am using the source version, I don’t have this issue, the source version takes long time to build for the first time and then it will never break.
switch between forward / deferred rendering (temporarily)
If you switch it back it won’t start compilation again.
Engine generally warns you about this stuff:
I’m not sure why many people here having issues with compilation of shaders. It is a lot of shaders, but it rarely takes more than a hour to compile. Even 30000 shaders. PCIe SSD + 16-32 threads CPU is a must for non mobile Unreal project.
Also disable all the options that increasing the number of permutations. Especially stuff like Quixel Master Materials.
Lol that’s nothing 8000 shaders.
The new engine has way better render, it’s set too a proper render level now so you get near photo quality graphics with the standard render.
I think it’s way better now they have raised the quality of the materials something i had to do manually in 4.
I reload 4 and it looks so bad now compared to 5, you might wait for shaders longer on the initial load in but once they have loaded they are quick to add into you’re levels and use.
I love the new render even if it takes longer to load the shaders first time round, 5 is such an improvement over 4 the only thing i’m having issues with in 5 is the DAZ3D bridge not working like it used in 4.27, the bridge is now broken in all versions because it fails to load into the daz3d Dim & central with a installation path like it used too for a plugin.
If you’re having a hard time processing shaders then you need a better pc, for the quality you’re getting with some textures it’s rivals even photo renders from DAZ3D that take hours too complete one frame where this is giving you fast frame rate per second & not losing much detail on meshes.
I only use version 4 now to convert project to 5 because it’s vastly superior particular with plugins added too it.
By the way i just loaded an old 4.27 project it had over 10.000 shaders so i don’t see the issue!
I feel sorry for you that you’re pc is slow and can’t render quickly but in the computing world you pay for what you get, you buy cheap you get cheap, you build cheap don’t expect many things to run good.
8000 shaders is something like a paragon from twinmotion takes about 10 minutes to fully render photolike.
Characters are %100 better and don’t take anything to load after initial shaders have been processed the first time in projects, they cache there after so it won’t take ages too load.
don’t get me wrong i paid big bucks for my pc, it’s 256GB i9 10th gen with a 12gb vram nivdia 2080ti so i expect the shaders to be loaded fast due to my ability to cache bulk memory.
It can take awhile if you have many projects that need caching to load all shaders, i always load overview maps after installing any new content from marketplace, that way the shaders load quick when you use pieces in you’re own levels.
I have over 600 gb of content in one project it took me about 2 or 3 hours to go through all of it & load overview maps to get the shaders cached but they all load instantly in project now, but it’s a massive 600gb of 4k & up textures is quite an amount tho not all would be shipped in the final game just a small portion…
this paragon is just over 5000 shaders took me around 8 minutes on my machine too fully render.
I have my ddc on its own drive
I know that each time I get new assets I have to open the map and let all the shaders compile
after that it won’t rebuild unless I change the platforms or the ogl/vulkan or desktop/mobile
or if you do something like activating deactivating virtual textures or mesh distance fields
I just installed the City sample
it had almost 80,000 shaders and it took 5 hours on my quad i7
I saved it and duplicated the main level and it loads in less than 4 minutes now for the small city level
I’ve loaded and made several edits and it hasn’t tried to recompile.
yeah loading just the basic 4.27 version loads around 6000 shaders it’s a little more with 5 because it renders at way better quality now that’s its worth using 5 for even with all the problems with bridges & content not migrating properly.
With a few bug’s ironed out of 5 i think it will be a solid editor overall but yeah you do need to set you’re PC up differently to the normal gaming machine setup.
When building a PC for rendering you need too have these in mind.
Powerful CPU with over 128 ram for cache, you’ll need to run something like windows pro 10 to have windows read anything over 128gb & 2TB HD’s on the system.
Vram good to have over the standard 8gb Vram in you’re video card, 12GB DDR6 gives a nice buffer so you won’t CTD due too being out of memory, but they cost big bucks.
Large system drive only for windows & caching, so you’re main drive C should be at least 2TB in size, you’ll need to plug other 2tb+ drives in for all other apps.
Example my setup is this for hd’s.
System drive for OS & Caching 2tb Samsung EVO Me.NV sdd
Epic Unreal Drive for Unreal projects only 2tb Samsung 970 EVO Me.NV ssd
Daz3D drive for the DaZ3D editor 2tb Samsung Sata 2.5 870 QVO
Games Drive for 1000’s of games & testing 2tb Samsung Sata 2.5 870 QVO
Can always plug more cheap QVO sata SSD drives in for more storage aslong as cache isn’t totally eaten up on C drive with all projects.
I’m running a quad 4th gen i7 with 16g and a 1060-6g
The city sample runs fine for me and hasn’t recompiled
Because an hour is a really long time to have to wait to start working with Unreal, and this is pretty much the first thing that happens when someone works on a VR project. Waiting for an hour might have become normal to Unreal devs used to the long compile times of the engine itself. But when you come from an engine like Unity, you can’t help but wonder what Unreal is doing after the first 2 seconds of shader compilation.
1 Like
I have found its better in ue5
every project has to compile the first time
each map has compile the first time
after that it should be fast if you save the build data
City sample took me almost 4 hours to do the first time shader compile
but it has textures that are huge
now it loads in a minute.
That’s the way it is if you want lots of cool materials
I start a new project that needs to compile, then I fire up blender and zbrush and keep working, they both will run together while the compile is happening so I can work.
I will also sometimes just open another UE window with another already compiled project
You can work on a second project while it compiles.
You can always choose your assets wisely for speed if you want.
I have some low poly stylized sets that load in a blink.
I was frustrated at first, but now I just accept that’s how it is
In the latest UE there is a new shader compile system that recompiles much less
1 Like
We’re talking about one time event. It happens ONCE. Ofc if you won’t change some fundamental switch, like adding Planar reflections.