I’m having some serious problems with shader compilation. I have been since UE 4.2 and it’s continued all the way through to UE 4.7. I’ve asked this question before but I’ve lost the thread so let’s start over.
A VERY SIMPLE shader is taking a VERY LONG TIME to compile. A shader compile value of about 350 has taken 20 minutes so far. I estimate it should take less than 30 seconds if everything were working right. I mean, it’s just plugging together some templated lines of code. I could literally write the changes in HLSL faster than this.
The auto-save feature kicked in during compile. This means UE is frozen. This is poor design and has to be fixed.
All I did was nudge a node’s position in the graph. This caused the entire shader to need to be recompiled. Seriously? Why?
No smart caching? At all? Even changing nodes should only recompile the flow beyond that point. Changing parameter values should not trigger an entire recompile.
I get that shaders compile near instantly for the Epic devs and most people but this process could still benefit from some optimization. Badly.
Anyway, I’m ready. Let’s solve this. There’s some stuff for you guys to fix and some stuff I can do. For starters, here’s my DXDIAG.txt: MEGA
Here’s a screenshot of my currently running processes. Yeah, that could improve. But shutting down everything makes little difference: Imgur: The magic of the Internet
I’ve editing one of my .inis after a support suggestion that I tell UE to hold back less cores for other processes. So it should be running on all four cores.
Compile just finished. It took 25 minutes. Here’s the material I was compiling: MEGA
Please download it, nudge it, hit Apply and tell me how long it takes you to compile it.
I’m running Windows 8.1, 16gb of ram, plenty of disk space left, etc. I’ve done all the normal things like kill pretty much every superfluous process and it’s still dead slow. I’ve spent thousands of hours over the last nine months waiting for this thing to compile. I’ve done my time. Help me figure this out and implement some of the suggestions above.
Figures. As soon as I post about it, it speeds the hell up. Now it’s compiling 250 items and it’s only taking a minute or two.
I shut down a few processes but the main thing I notice is that although I cleared out maybe 50% of the background task load, the machine is still only sitting at 50% usage. So it could definitely be going even faster.
The shader workers actually show in the process manager now but they’re only going at about 7% each. The parent UE process is using about a quarter of the available CPU time. What’s it doing? Is it compiling, or is that just idle IDE usage?
The other problem is, the change I made to make it compile was to move the main attributes node slightly. So this really could be improved as well.
So items 2, 3 and 4 still really need to be looked at. Item 1 is still not ideal but not killer either. Tomorrow it’ll be back to crawling I’m sure. Go figure.
While I doubt this will help you much, I tested your material and for me it pretty much compiled instantly (25 secs I guess). If I move a Node and save again, it does not recompile.
While it’s true that freezing the engine during autosave is not the best behavior, you can just turn off autosave and you’ll be fine.
This is so bizarre. Why am I the only one getting full recompiles? And I need autosave - UE isn’t exactly stable so it’s nice to have. I don’t think we should compromise on this.
Still, that’s more helpful than you realise! What CPU and how much RAM do you have Deventico?
Let me begin by saying that I have entered feature requests for our engineers to look into adding a test to see if the compiler is running to delay Autosave accordingly (UE-11362) and to separate Shader Compilation from Material Node Location Saving (UE-11364). I will keep you informed when I know more.
1 - There is a lot of Shader Code which is compiled to ensure that the Material is both optimized for your look and for performance in the engine and ALL of the code is checked against your Material node code snippets each time and rewritten as necessary, you can review it by looking in the HLSL Code Window. Having said that, If you can see anywhere specific that we could save some compilation time, please let us now and I will submit a request to have our rendering engineers review the change. We are always eager for users to help us improve and make the engine better.
4 - You can use a Material Instance instead of changing the Parameters in the Base Material and you should have instantaneous feedback without the need to re-compile. Using Material Instances in this way is our own workflow.
