UE5.1 First start after installation (Preparing shaders....)

Development Getting Started & Setup
,
21

Well, it’s a bit different on my end :smiley:

This Appdata cache (which imho should never have been designed to be generated inside Appdata in the first place, since the Engine is supporting custom install locations other than C, and Appdata is supposed to only be for light portable user and app settings but not actual content) is reaching 13Gb here, probably because of the various versions I’ve used. This right off the bat is awful design.

And then (assuming that this cache is supposed to take care of the post-launch/per-project “preparing of shaders”), I unfortunately can’t seem to find a way to properly tell the engine to use it as I am still getting some “Preparing shaders” wait time once some projects are opened. Or perhaps the path was correct, but the contents corrupted ? No way to know !

At this time here’s all I know :

  • The compiled cache (that ships with the engine) seems to need to be linked in the first entry of the cache settings, like here :

image
image697×165 3.9 KB

This seems to consistently get rid of the shader compilation wait time on editor splash launch (when the Unreal logo and purple landscape image show up, before a project even opens up). But of course this setting won’t help anyone getting that compilation wait time anyways … since one needs to reach the editor in order to fill that up. Unless users are expected to know which .ini to tinker with in order to link it up, before even starting the engine for the first time ?

• The last two entries are still 100% mysterious to me (not in terms of what they are supposed to do, but in terms of how to fill them up) as I am still sometimes getting a “preparing of shaders” while a project is loading up. I suppose these caches are supposed to contain the compiled shader instructions specific to a project, but why would there ever be that many - almost as much as the engine itself ? If they are that important, then the engine should just gradually compile them for each project and put them all in common folder inside its install location, or inside a user-defined custom folder explicitly asked about in a popup when opening the editor or a specific project.

Could someone from Epic please clarify this once and for all ? There should never be a need for a 30+minutes wait when opening a project (even the most complex ones ever), and the docs should explain everything much more thoroughly.

And, if a content pack requires that many extra shader instructions to compile, then there should be an option to download the corresponding cache.

And even more importantly, all compilation wait times should be skippable with a button in order to be able to reach the editor settings.

What are the various meanings of the little cache icon on the bottom right of the editor anyways? Does it need to be green ? And when it is, what does that actually mean ?

3 Likes
22

Same problem here. I used 5.0 without this problem before. I use a custom install location but that wasn’t a problem in the past. After updating to 5.1 i had no problem when opening a EXISTING project.

Creating or opening a new project it always starts to rebuilding / preparing the shaders again and again. Any news on that?

1 Like
23

This bizarre behavior seems to be carrying over to packaging as well.
In 5.0 the packaging process is proceeding without any issues, in 5.1 however, it gets stuck at this line for a good 10-15 minutes.

UATHelper: Packaging (Windows): LogCook: Display: Checking global shaders for platform Windows

I wonder if this is linked somehow. Projects created in 5.0 and opened in 5.1 are not affected, but newly created projects in 5.1 just refuse to use DDC that it just compiled. Packaging a map with 10 static meshes took 30 minutes =-=

I’d rather not melt my CPU by compiling default shaders.

Edit: subsequent builds didn’t trigger any shader recompiles.
But enabling Virtual Textures in settings did! >.<

24

Okay, some more googling seems to have shed at least some light on why it recompiles.
This topic from 2 years ago covers pretty much the same issue and apparently enabling Ray Tracing or any other major rendering feature forces shader recompile.

Thus, if you create a project with Ray Tracing enabled, it will have to recompile all the shaders, including the engine ones. At least on my end this seems to be consistent:

  • Creating a project with either scalability setting with Ray Tracing enabled always triggers 7k+ shaders compilation sequence;
  • Creating a project without Ray Tracing ticked launches it instantly;
  • Changing anything like Ray Tracing or Virtual Textures in a project created in 5.0.0 also triggers shader recompile.

Now the question is, why do we not have a precompiled set of Ray Tracing shaders? Is there some arcane technical limitation which makes it not possible to ship such a package in addition to the non Ray Tracing one? I understand that you can’t have all permutations compiled and served to you, but could we at least have the most common set of settings precompiled? I’m genuinely curious since I didn’t bother with Ray Tracing at all before this year.

For now I guess the solution would be to just create a project without Ray Tracing ticked on, setting up all rendering features that might trigger recompile (these usually prompt for engine restart) and then patiently waiting until shaders compile, since this will not happen again unless you toggle a major rendering feature.

Also, don’t forget to package early. It checks the shaders on the first packaging.

25

Because it’s wasteful. There are a small number of engine shaders in total and they typically compile in a matter of seconds. They are compiled locally for the target platform/hardware/settings you are using - rather than the alternative which is to download a huge library of permutations for all possible platforms and settings, most of which will never be used. Not to mention the additional book-keeping required to ensure you use the right ones at runtime.

Platform-independant data like textures and mesh data are included in the launcher builds’ Derived Data Cache IIRC - but it does increase the download size.

Your project will make up the vast majority of total materials/shaders you need to compile anyway eventually (and obviously they can only compile locally). I don’t think anybody should be toggling major rendering features on/off at such a frequency for this to be much of a problem.

Since Ray-Tracing is disabled by default so when you launch for the first time, shaders are compiled without those permuations. You can skip this initial compile by enabling Ray-Tracing in the project *.ini before opening it to avoid compiling all shaders twice. If you’re experimenting, use a small project with a limited number of assets before applying it you your flagship.

26

Im new to Unreal 5, but your posts helped immensely!

From the documentation it says to put this line of code in the DefaultEngine.ini located in the project’s “Config” folder.

[DerivedDataBackendGraph] Shared=(Type=FileSystem, ReadOnly=false, Clean=false, Flush=false, DeleteUnused=true, UnusedFileAge=10, FoldersToClean=10, MaxFileChecksPerSec=1, ConsiderSlowAt=70, PromptIfMissing=false, Path=\YourCompanyServer\DDC, EnvPathOverride=UE-SharedDataCachePath, EditorOverrideSetting=SharedDerivedDataCache)

Of course changing the path to where you want your shared DDC to be. I didn’t have to make it mapped either. Its just on my C drive. Hopefully this doesn’t cause problems. I’m just working alone anyways.
It also lists 2 similar ways of achieving this by editing a environment variable and your method of going into the settings. It says editing the .ini file is the more preferred method of making a shared DDC.

So far I don’t see the annoying 6K+ :nauseated_face: preparing shader stall on open or when I access something I already loaded shaders for. Hope this helps!

27

Update on my issue:
The problem was gone the next day after i did the update to 5.1. Not sure why, i didn’t change anything. So for now everything works fine again for me.

→ Update: Sorry, i mentioned the wrong version number. I am using 5.0.1 and this is the version i got the initial problems with. I cannot find any 5.1 at the moment so maybe articles writing about a 5.1 are not using the correct version number.

28

Are you talking about 5.0.1 or version 5.1.0? I keep seeing articles on google talking about UE5.1 as if it is officially out yet and its confusing the mess out of me.

29

Sorry, i took the wrong version number after reading all those articles.
I am using 5.0.1:
grafik

Think a 5.1 is not officially released yet (at least i couldn’t find it). I agree with you that it is very confusing so much people writing about a 5.1.

One thing i found out is, that changing the raytracing settings (e.g. support hardware raytracing) leads once to a shader compile / recompile.

1 Like
30

Ok thanks for clarifying this!

1 Like
31

I’ve modified both the BaseEngine.ini and the ConsoleVariables.ini as explained here, and that solved the shader compile at startup problem, but opening any project triggers a full shader recompile - 8k or so - and that makes UE 5.01 unusable.

It’s a drag.

32

Same freaking issue here. Even with all the temporary workaround, I found in this forum, the only thing I can do is just close all and any programs/process (nonsystem ofc) and wait. it just takes so long.

I just wanna start learning and using this brilliant software, but it just keeps giving me a throwback to the days I used to render archviz with v-ray on a $200 laptop this past week.

computer spec: a 1060 laptop with 32 gigs of ram, the rest can be easily figured out.

33

Yea, with the recent Unity stuff I wanted to try a sample project with UE5.

Completely fresh installation, created the VR project and both starting the project and attempting to run it resulted in preparing 5000+ shaders. With the other projects on the go, I’m tired of my tools breaking on first run, I’ll give UE5 a pass for awhile if this is the state it’s in, I don’t have time to fix someone else’s work.

34

Hey, all! Is there a solution to the topic? I’m thinking to uninstall UE5 now and come back to UE4. Its impossible to waste so much time every time I start my project.

36

I’m in a similar position deadend with UE. I start a new project with ZERO content, no starter content, etc. It spends 20-30 minutes to load 8000+ Shaders, and then once it completes and I’m finally in the editor, it wants to load those 8000+ Shaders all over again for some reason. (At least I can begin working at that point, but why on earth does an empty project take soo long?) Is there any solution to this?

37

Can you please clarify on this? I added the strip of code, entered a path for the DDC and so far nothing has changed at all.

38

Does anyone have an answer to this? Because i had the Issue with 5.0.3 but somehow by turning off lumen and nanites i got it working fine.
But now with the new 5.1 version i have the exact same issue again.

It loads about 7000 shaders every time i open a project, empty or not. Even a 2D PaperSprite project.
Takes forever to load and even with Lumen and Nanites turned off, it still loads 900+ shaders. And those take also easily 5 minutes to load.

40

I tried all of the fixes in this thread and its still happening for me. I’ve wanted to dabble in UE for years but every single time i try and do anything with the engine ever this happens. I think i’m just gonna give up on it at this point :confused:

1 Like
41

Did this work? Because I’ve been building shaders for quite some time now.

42

@Numt3k : as of 5.1.1 the problem is very much still there. The biggest shader preparation seems to only happen once, but the engine is still designed in such a way that it requires unacceptably lenghty shader preparation for each project, and even when reopening an existing project.

And of course there is still no clear word from Epic on how to mitigate that. They probably don’t experience the issue the same as we do, has their IT probably deploys a precompiled cache to their machines and they also likely also don’t install the engine the same way as we do. This is all conjecture of course.

For anyone who needs speed of iteration, the only sensible way is to develop under 4.27 IMHO. That’s personally what I do. The 4.2x editor is just faster than 5.x at pretty much everything anyways, and the UX is superior overall (no “sliding dock” getting in the way).