Losing hours and days to re-compiling a project when making extremely small changes.

Development Programming & Scripting C++
1

So far I keep encountering insanely long compile times for a basic, barely put together project.

For example here are the following issues that I encounter which ends up resulting in 5-10 hour re-compiles.

  • Deleting a c++ file (in this case include cpp/h)
    • Results often in a total re-compile of the project for unknown reasons. Even when the file wasn’t being used by anything at all.
    • Usually thousands of files.
  • Moving a c++ file
    • Results in a total re-compile of the project, even if the file wasn’t in use anywhere
    • Usually thousands of files.
  • Creating a c++ file
    • This usually isn’t too terrible and the IDE & UHT seem to figure it out. However what happens is;
      • Make the class in UE
      • IDE pops up & ‘live coding’ I guess does a small compile.
      • Then you need to refresh the project in the IDE and close it.
      • Close the editor next
      • Re-open IDE and make your actual changes
    • Re-compile project (THIS in my experience is the ONLY short-lived compilation, meaning it’s usually only a few files, not thousands.)
  • Adding a component via C++ to something that already has a blueprint child in-project
    • It appears that UE doesn’t ever pick up that you’ve added the component
    • Results in wiping everything already compiled in the project trying to trouble shoot why UE doesn’t see the component
    • Thousands and thousands of files being recompiled. No idea why.

So here I am, once again just trying to troubleshoot why UE doesn’t see anything as if it doesn’t exist. That process thus far was;

  • Delete the Binaries folder
  • Delete the Intermediate folder
  • Re-generate the project files
  • Run Compile on the project in my IDE

The new total is now 4205 files. Look, I’ve basically written around 20 files, not thousands. Last time it did this it was around 3100 files or something.

I’ve set the compilation system to use every available ounce of 64gigs of ram and all of my Ryzen 9 processors.

Still I am looking at hours and hours waiting to see, if UE can actually see anything. It’s rather mind-boggling to me to think that anyone, in this day and age, would expect to work like this. Is there some secret super computer that studios have that magically compiles things faster?

Why would my project’s files for compiling magically bloat by a thousand more, when I’ve not added any to that scale?

Watching the compile as I write this, why is it re-compiling things like UnrealEditor-ConsoleVariablesEditorRuntime.dll when I’ve not made any changes to that or any of it in any shape or form?

To me, it looks like it decided to recompile the engine itself, which shouldn’t be the case as I specifically told it to compile the project. Also, the engine itself if I were to recompile it, would be 12000 files and take 24 hours.

I guess I just don’t fully understand the scope here;

So I’m compiling a ‘project’ version of the ‘editor’, but why and how would it change from 3000 to 4000 when I’ve done literally barely anything since the last project compile? Usually my project actually compiles about 1000 files if I delete the binaries & intermediate folder. Which fortunately only takes 30 minutes.

Why am I forced to delete the folders, does UE have that much of a dog-biting-bone issue with ‘ghosts’? Sure seems to.

I guess I don’t understand why when I literally define something specific the UE can’t seem to understand it unless I wipe from orbit and rebuild from scratch. It seems very silly to me.

2

i can add a c++ file, recompile and be back in editor within 5 minutes. My mid range dev computer is about 6 years old. Something is wrong with your setup.

When adding a file thru unreal, give Visual Studio a bit after unreal sets off the compiler and see if it asks to re-load the solution. You shouldn’t need to re-start VS adding a file.

Deleting your Binaries/Intermediate folder will force recompile all your project objects, shouldn’t need a recompile of your engine.

Check thru your build.cs and .uplugin files to make sure they make sense. Try setting up a super basic c++ project and see if you have the same issue there.

Windows system and driver updates can cause a recompile of the entire engine. That gets me every once in a while.

The problem with adding a component to a pre-existing actor in a level is the way unreal saves objects in a project file, when it reloads and tries to re-map the data it fails and can cause all kinds of issues. If you spawn your actors/components dynamically this isn’t a problem. This is just something to be aware of and to work around. TBH this same issue cost me half a day last week even though I have run into it many times before.

3

I hate to say that this is either a local setup or a user problem because neither I, not any of my colleagues have ever encountered this type of behavior when building projects. Perhaps a few pictures of the UI that you’re interacting with in your IDE. And of the Solution Explorer window for your project.

How are you choosing to build when you do it? From Visual Studio, there are some regular C++ options that should not be used when building Unreal Engine projects due to how UnrealBuildTool is invoked.

Neither of these things should cause a re-compile of the project. They may recompile more than you expect sometimes (due to the Unity build process) but not the whole project.

This whole workflow is bad. For an Unreal project with C++, you should start with opening up the project in your IDE and running the Editor from there. You’ll still have to close the Editor to add files, but you won’t re-open the IDE every time you make C++ changes. Ideally you should also just make C++ files with the Editor closed. You don’t need to do that through the Editor.

Yeah, that shouldn’t be happening. As long as you’re creating it correctly as a default object, it should get added to existing blueprint children just fine. It always has in my experience.

Yeah, unless you’re using a source build and making changes to engine header files you should not be compiling this many files on a regular basis. It feels silly because it’s not supposed to be working that way. No one else complains because it’s not doing that for us.

4

I take it you’re building from source. Editor projects used to default to Shared. Now they default to unique. That’s the issue. It will rebuild the entire source to match your settings. This will happen regardless for a published build. But for a build that you run in the editor, you don’t need that. Go into your editor’s target.cs file and add this line:

    BuildEnvironment = TargetBuildEnvironment.Shared;

This should fix your issues. You may have to delete your Binaries, DerivedDataCache and Intermediate folders. You’ll have to right click on your uproject file and regenerate project files.

I’d delete the Saved folder as well. You can keep any actual save game files or render queue files in there.

This will cut down your build times to seconds.

One more thing. Make sure your commit size is at least 3x your RAM size. So if you have 64GB, at least 180GB of virtual ram on your C drive is necessary. This is what is used to calculate how much ram is available for each core when building and you want to use all the cores you can. And no, it won’t go into virtual RAM if you have 64GB. It’s just stubborn in its calculations.

1 Like