Building the Engine from source. Following this article.
Says there “it may take between 10 and 40 minutes to compile the engine” - yeah, I wish!
Total build time: 7473,02 seconds (2 hours). No errors.
My Specs:
i7-5820K @4GHz (6 Cores / 12 Threads)
32GB RAM (Quad Channel)
Samsung 970 Evo 500GB SSD (NVMe)
Windows 10 64bit
Visual Studio 2017 CE
CPU load on all cores was between 10-60%.
Is this a result within norms, i.e. to be expected with the first build despite what the official article says, or do I need to check my configuration?
It does seem a little bit high. On our build server, a full rebuild takes 1 hour and 5 minutes. I see similar loads as yours. That machine has a somewhat weaker CPU and a normal mechanical drive, though.
On my development machine the load is much higher (80-90%) and it’s somewhat quicker, 40 minutes. Pretty much the same setup as yours but double the RAM.
I think it’s safe to say that the article is referring to partial rebuilds.
I’m especially annoyed by the low system usage. What point in having a 4GHz hexacore if it idles away on all threads, not hitting anywhere near 100% on any single core during the entire process? Nothing else strikes me as a clear bottleneck, either, not RAM nor disk I/O.
Any particular settings to get VS to utilize more resources?
Maybe it’s not using all the cores. You can change the UBT settings to make it use more simultaneous processes, by editing the BuildConfiguration.xml and setting MaxProcessorCount to 12 or something.
Also, keep in mind that many steps of compilation and linking are very IO intensive, so the SSD can become the bottleneck on a very fast CPU.
In general, sure, but in this particular case - where the CPU is almost dated while the SSD is near as fast as you can get today…
While observing the performance monitor I didn’t notice any significant load on the drive.
I’ll definitely give the BuildConfiguration.xml a try - though, all cores *are *at work evenly already. The CPU just doesn’t seem to get much to do. No idea what the bottleneck could be.
With hardware like this, 2 hours is definitively too long.
On my old Xeon E3-1231v3 (4c/8t, 3.4 GHz) it took 46 minutes after I put the output folders on my SSD. I upgraded to a Ryzen 7 1700X (8c/16t, 3.7GHz) and got it down to 31 minutes. Then I moved some more folders to the SSD and got it all the way down to 25 minutes.
You should update the BuildConfiguration.xml, but don’t adjust MaxProcessorCount and set ProcessorCountMultiplier to 2 instead. Since you have hyperthreading, you can run two threads per core. My BuildConfiguration only changes this one setting:
If you aren’t getting constant 100% usage, the multiplier is likely the issue. You have to manually set it to 2 for some reason (it would be really easy to auto-detect)
Also make sure your memory is running at the correct speed, if it’s using XMP it may be set far too slow by default
Turns out, it was IncrediBuild that caused the problem.
Ironically it is supposed to accelerate compiling, but it slowed it down for me by a factor of 4.
Without it, CPU load hits 100% on everything almost constantly, rather than the stop&go before.
PS: It kept telling me, that I didn’t have a license - where would I register it?
Use FASTBuild instead. It’s free and open source. It allows you to distribute compilation over a network and use caching. If you only have one powerful computer, the former isn’t relevant to you. The caching is great though, if you can sacrifice 100 or so GB of SSD space. The first build is slower (because the cache needs to be build), but rebuilding after that is a lot faster.
Oh, that looks very interesting. What exactly does this cache do? Will it for example speed up unnecessarily rebuilding the entire engine after changing a header file?
I often have cases where all files need to be rebuilt since a core header got changed but that change is not actually affecting more than 3 lines in one specific cpp, it’s infuriating
From what I understand, it caches the compiled object files on disk and hashes them in some way. When you recompile, FB can use that information to copy the already existing object file rather than recompiling, if the file is compatible.
How much has to be rebuild depends on which files you changed and the nature of those changes. I haven’t tried, but I wouldn’t be surprised that you would have to recompile a lot, if you started doing some massive refactoring of Actor.h or some other comparable header file.
The only engine changes I made were in World.h and PointLightComponent.h. I added functionality to both of them and the recompiles were blazing fast for how deep inside the engine the changes were (<2 minutes). I mean, we’re talking about World.h here, that’s included/used in a lot of places.
One thing worth noting is that which compiler you use is a factor for whether a cached object gets invalidated. That’s important because Microsoft now releases tons of small updates for Visual Studio on a regular basis. If they update the compiler, chances are your cache will become invalid.
Got around to checking out fastbuild, the caching doesn’t actually help when you are changing things. If you save a core header without modifying any code, it will save you, but that’s all it can do.
It’s crazy that it takes so long. I have about the same PC specs and it takes less than an hour. I’m using Incredibuildso that is probably the reason, but still, it doesn’t make sense. You could try it out if anything. It’s easy to install to VS.
I have a Dell G7 16gb ram and intel core i7 8th gen and yet it took 8 hours for just to build the ue4.snl file. any tips to make visual studios faster?