First, sorry if this is a newbie question or if there’s another thread for this, I tried to find one but couldn’t. I’m using UE4 for a couple months now and I’m a designer, not a coder, so I love Visual Scripting but I am also fascinated by coding. I’m trying to understand specifically how do several size teams deal with C++ compiling times. I’ve heard in the forum and AnswerHub some people saying their small projects take like 30+ seconds with each compilation. How does this scale to bigger projects? How to manage that?
As far as I know, if you have to wait this time everytime you wanna build, you either won’t test your build very often so you don’t have to compile again or you’ll be waiting and waiting like you’re playing Clash of Clans hehe. How do you guys manage compiling times? It would be nice to have someone from Epic talking about this issue in bigger games like Gears or Paragon.
Complex games can take a while to build. My advice: keep distractions at the ready. There are days where I spend more time waiting for compiles than I do coding.
So that means that every time you code anything new, if you want to test it out you gotta wait for the entire compilation, which can take from a couple seconds to several hours depending on the size of your project?
Do you guys consider that a big problem, or it’s just an inconvenience?
If that’s really a big deal (which is currently sounding like to me) it raises me a question: Why to choose that? I’ve already read a lot of the C++ vs C# threads and such and I saw there are many arguments both ways… but not-having-to-wait-so-long vs having-to-way-a-lot looks like a very different deal to me…
Unity is leagues ahead when it comes to iteration.
In Unreal my partial rebuild times are about 20s, full rebuilds a minute or so.
Don’t forget the time it takes to restart the editor if you crash it, then compile a debug version of your dlls, get Unreal to hot reload them, attach the debugger and finally break on the trouble-line-of-code.
That super long cycle makes it really tough to stay in the zone sometimes. It absolutely kills my productivity on some days.
So yes, the iteration in Unreal is inconvenient. Personally, I pay the iteration tax because I think that Unreal is the better engine. It has more features, full source code access and C++11 is the faster (and IMO better) language. Finally, having access to scientific code libraries written in C++ (Eigen, libigl, Suitesparse, Cholmod, CoinCLP, …) is absolutely critical for some of my projects.
Separating game code into multiple game modules enables faster compilation - only the modules that change need to be recompiled.
You are only compiling your own changes until you pull from the repo. Thus, you are in control of how often you have to recompile everyone elses changes.
Prototyping in blueprints (or keeping certain features in blueprints) eliminates compilation.
Something like IncrediBuild can distribute compilation load across all machines of the team.
I use a continuous integration (CI) service (Jenkins and Vagrant) that is plugged in to my Gitlab server. As we do pushes to the various branches, the continuous integration server will do the compile and run some tests. We normally push our branches when we are satisfied we have a stable feature.
Most of the work is done inside Visual Studio and the editor and the CI service handles everything for me. This includes doing Linux Client and Dedicated Server builds as well as the Windows Client and Dedicated Server builds. I haven’t managed to get the Mac side of things working … but we are okay with doing manual builds of that.
Nice. So, how would separating modules working? What kinda modules would you have? One for each game mode, something like that? One for each level?
Regarding Blueprints, which doesn’t have this compile time problem, how do you guys go about it in mid or big size projects? What’s the sweet spot proportion between coding in BP and in C++? You still do most of your project in C++ then? Wouldn’t it be possible to do a good chunk of it in BP and avoid the compile time?
That looks like a lot to deal with. What do you think it would be possible to do to make it better? Or by the nature of C++ is simply not possible? Did you see something in UE4 Roadmap that can make this more practical?
The granularity of module separating depends on project and team - for example, you could put use a reward/progression module, vechicle physics module, etc…
The cost of using more modules makes inter-dependent references harder - this is the trade-off.
Bp / c++ division is also project/team dependent. BP is great for artists, great for prototyping, and easy to change/enhance. There are some things that must be done in code - although these are getting fewer all the time. Seems to vary greatly between developers - some embrace bp others seem averse.
One more thing - on a large team, you can include binaries in the repo so that artists and BP only users do not need to compile at all.
Highly recommend Perforce over Git for a large team due to locking of binary assets.
Also have not seen it mentioned, if I need to test some logic specifically I put it seperately, test only that function and then compile just that. (like in its own project) - consider it like little tiny testbeds to test certain functions etc
If that works then I move on to implementing into main branch, here you just will need to wait I am afraid and then see what you just broke Also before I forget - there is RECODE, I am using that with CRY and saves a lot of time. Google it.
Hmmmm that’s interesting. But how can you put it separately in its own ‘project’ if it needs to be at least together with other parts of the gameplay in order for you to test it? You mean if there’s this jump function you are implementing, you put it separately together with all the rest of the code needed to support that feature?
Yeah thats the gist of things, yes you can argue that it might slow general workflow down, but at the end of the day if I want to test AI behavior do I really need all those textures and maps and god knows whatelse you might have as your map/game logic? Look at an example on how StarCitizen is doing it, from looking at their Around The Verse videos when they show the teams working on different aspects of the game, all developers are using the base code of the current dev build but run their own test maps to test out their own specific things they are working on, in GIT essentially on their own branch. Then once a feature is ready for implementation it gets implemented into main dev branch, then handed to QA to test, then back to bug fixing, then tested again until fully working as per SCOPE, and then merged with main branch and deployed to all others as base.
Remember this is very simplified. But you get the idea. And YOU are the one who knows what is required or not by that specific module. Thats why a good and well commented Design Document is of utmost importance, even more important the bigger your project goes. So ultimatively you are the one deciding what can be developed outside of main project and then later merged into it.
Here is how I do things:
Prototype -> test -> fix -> test -> merge into main dev -> test -> fix -> test -> publish onto master — rinse and repeat until all features are implemented
Yes this might take more time at the end but at least you have a clean flow and testbeds for certain things later on which you can now extend.
The network game I am working on at the moment is getting quite big with all the subsystems and functions in place that I am currently sitting at around 1-5min compile times with quite a beefy PC. So it is something that unfortunately you’ll have to endure through I am afraid. Get side activities at the ready or go for some fresh air while compiling etc Nothing beats a fresh breeze.
And the above doesn’t just apply to game programming, these are workflows you can utilise for all sorts of projects.
Nice! In which Around the Verse vídeo do they point this out specifically, can you remember? I’d like to check it out.
But your argument raises again the same question for me: maybe someone who already has experience working with C++ and compile times would rather just keep working the way he/she does instead of bothering changing anything, but for a newbie, why would this be a good tradeoff? If performance is the big plus, honestly I don’t know, most people complaining about performance in Unity/C# are people here in Unreal forums hehe. I believe both engines/languages can run into performance issues if not correctly optimized, and with tech advancements this becomes less and less an issue especially if we’re not talking about the biggest and more complex games, or am I mistaken?
What would be the other points to answer the question: Why would a newbie choose to deal with compile times when they could choose not to (by choosing a different language/engine)?
None in particular really, just looking at their setup and I can tell you with a high amount of certainty that this is how they work. Otherwise how do you imagine would they be able to split all this work between the studios? I remember Chris mentioning as well (some time ago) that it takes them 3+ hours to compile everything. Then again they have DevOps engineers etc that take care of automation and build routines / cycles etc. So like I said unless you have a big team and the monetary resources to support it, there is no way around wait times, just clever (#lazy) engineering to do only the necessary bits in their own environments.
To be honest, everytime I compile Cryengine I cross fingers haha And let’s not get into the conversation around Unity or C#. While Unity might be good to learn initial concepts many of the long term programmers that have worked in C++ and written custom engines etc will tell you that while it is “cool” you are not as free as with Cry or UE. Also if you want graphical fidelity w Unity you simply cant get it (unless you’re willing on spending hundreds in plugins and then hoping that they all work together.). Unity was my goto engine for Android mobile games, but since Unreal and Vulkan are a thing I am switching even that stuff over as well. Unity in my humble opinion still needs to grow up.
And to answer your last bit there… well no one is forcing anyone to use Unreal. And if you are a “newbie” to C++ etc, I do not believe the right point of entry should be the engine but rather a basic learning course in C++ itself to at least understand all basics, including Pointers, References and inheritance & structures. Once you get the basics down you can start looking into Unreal programming. Until then Blueprints are a great way to learn When I started back in the 1993 I came from Ansi C and was using Borland C++ (you can google that one ) writing console applications for fun and school projects. Or just using asm {} to hack the schools printer etc. Name me a AAA engine that is open source like Unreal or Cryengine with the same fidelity and options… I looked into many, stuff like Irrlicht etc comes to mind, while they all tried to do something specific and lacked versatility at the end of the day and you could only make a certain type of game, while with Unreal and CryEngine you could go and be creative. Ok Unity as well
This one might be too obvious to mention, but think and plan a little bit before messing with header files. The more files include your changes, the longer it will take. Try to leave the rapid iteration to the CPP files.
For AAA sometimes classes are growing so fast that they reach enormous levels. I discovered that some crucial gameplay classes like Pawn have actually 120 000 lines of code. They are divided into separate files like Pawn01.cpp Pawn02.cpp … Pawn12.cpp because there is 10k lines of code limit for each file. Not sure who started that but it happens.
So just think forward, divide code into smaller classes (managers, factories, libraries, components).
Try to reuse code as much as you can with static libraries / helpers.
Make new modules for independent code (Audio, Gameplay, Input, Interface, Session, Online, Types, UI, Utility, Event, Extension, LifeCycle etc). In the end having more modules in your project helps.
Use interfaces to communicate and to not creating additional dependencies
IWYU - include actually what you use, not more, even CoreMinimal includes a lot of libraries.
Incredibuild helps but what if your project looks like /UE4 Engine itself?
Also before I forget - there is RECODE, I am using that with CRY and saves a lot of time. Google it.
Nothing beats a fresh breeze.
) writing console applications for fun and school projects. Or just using asm {} to hack the schools printer etc. Name me a AAA engine that is open source like Unreal or Cryengine with the same fidelity and options… I looked into many, stuff like Irrlicht etc comes to mind, while they all tried to do something specific and lacked versatility at the end of the day and you could only make a certain type of game, while with Unreal and CryEngine you could go and be creative. Ok Unity as well 