An interesting strategy to counter ‘long’ build times (which are definitely improving) and boost programmer productivity would be to allow users to compile projects using distibuted compilation ala the Swarm Agent, but for C++.
I know that there are tools available that can compile in this manner, such as Incredibuild, but it could be a great feature to add to UE4 and its custom build chain (UBT), and thus be included in the cost of our subscriptions.
There are certainly times where I wish I could utilize the power of my several PCs at home, or the 10+ computers at work that run UE4/VS2013.
Considering that Epic’s UBT is run whenever we compile, it seems, from the outset, that they’re in a good position to create such a thing.
This would certainly be advantageous for full engine recompiles. Not sure how much it would improve productivity really (given how much time is spent in the initial setup versus the actual compile for a single project), but it would definitely be a nice thing to have.
You have to consider the initial setup as a static cost vs. the pleasant workflow of continuous changes that in the end will simply make a better end product and that you will benefit from each day. If you’re working on a game the initial setup of perhaps a day depending on your technical background can easily be recouped over the year+ development cycle.
I just set up IncrediBuild on two of my machines as an experiment for full ue4 compilation. Next step is to hook up Amazon AWS and see how that cuts down of my compile times.
Finally with the improvements Unreal is making to hot reloading it may become less interesting over time, but I expect there will be a use for distributed building (light baking too) for a long time when working on small to medium sized teams.
While typing this the first UE4 4.5.0 build completed and the results are:
**981 seconds distributed over two machines 8+8 cores vs. ~ 1900 seconds on local machine previously! **
When I say initial setup, I’m not talking about setting the system up, I’m talking about prepping for compile. There is a point where the actual act of building the project normally takes less time than the overhead of the distributed build, which is why I’m saying engine recompiles would be a definite win, whereas recompiling your game project likely would see no benefit at best (assuming the system could detect when dividing the build would not be worthwhile). The question is, how frequently do you need to rebuild large chunks of the engine? If the answer is quite frequently, there’s a good chance you’re probably doing something foolish (frequent changes to shared headers?). I’m not saying it wouldn’t be nice to have, and I certainly would support it, but the productivity gain seems minimal.
It would also apply to packaging a project, which works via UBT as well. Some of us re-package quite often to test things, and spend lots of time waiting for compilation (even when just compiling some game code).
Considering it would be working over LAN, the transfer, at least, wouldn’t take much time.
I’d be very interested in seeing Epic’s opinion on this, regardless.
I misunderstood. I don’t have any figures on setup times and compilation boosts for an avg. ue4 project, but I can say that when working on a project like ShooterGame (which yes, does have very inconvenient header setups for sure) you can notice a decent improvement on just adding one additional machine to compile. When you’re working on larger projects and many coders I cannot believe it would NOT be beneficial to have a shared pool of CPUs to constantly benefit from. Still, you are completely right on keeping headers clean and single machine compilation to a minimum (Epic is doing a great job with this since launch)
I have a similar setup (one laptop and one desktop) and it was about 12 minutes for me. That’s quite amazing, the engine is over 2 million lines of code! Have you seen that building the engine from source performs better than precompiled binaries from Epic?
