Compilation taking twice as long in UE5: Number of processes being limited?

Development Programming & Scripting C++
, , ,
1

I updated my project from version 4.27.2 to 5.0.0 and C++ compilation is taking about twice as long, sometimes longer. This occurs for both compilation from Visual Studio and for Live Coding.
Looking at the compilation logs, the main difference appears to be that the number of processes is lower in UE5. Refer to the below logs:

Visual Studio Community 2019 16.11.12, toolchain 14.29.30142
Windows 10.0.19041.0 SDK
Intel(R) Core™ i7-6700 CPU @ 3.40GHz (Cores: 4, Logical processors: 8)

Test case: Adding and removing a whitespace from a relatively small cpp file in my project and compiling.
Note: This is a project-only build configuration, so there is no Engine compilation involved.

Live Coding, UE4:

 Building 1 action with 8 processes...
   (Module.proj.cpp)
  Total time in Parallel executor: 43.87 seconds
  Total execution time: 44.67 seconds

Live Coding, UE5:

 Determining max actions to execute in parallel (4 physical cores, 8 logical cores)
    Executing up to 4 processes, one per physical core
  Building 1 action with 1 process...
    (Module.proj.cpp)
  Total time in Parallel executor: 92.17 seconds
  Total execution time: 97.24 seconds

Visual Studio, UE4:

1>Building 4 actions with 8 processes...
  (module.proj.cpp, UE4Editor-proj.lib, UE4Editor-proj.dll, ProjEditor.target)
1>Total time in Parallel executor: 45.47 seconds
1>Total execution time: 46.32 seconds

Visual Studio, UE5:

1>Determining max actions to execute in parallel (4 physical cores, 8 logical cores)
1>  Executing up to 4 processes, one per physical core
1>Building 4 actions with 4 processes...
  (module.proj.cpp, UE4Editor-proj.lib, UE4Editor-proj.dll, ProjEditor.target)
1>Total time in Parallel executor: 86.94 seconds
1>Total execution time: 87.70 seconds

As you can see, in addition to taking about twice as long, 8 processes are in use when compiling in UE4, whereas either 1 or 4 processes are in use when compiling in UE5, for Live Coding and Visual Studio respectively.

Is this issue occurring for anyone else, and does there happen to be a workaround or is there a fix in the works? This is a pretty significant hit on productivity, especially with my 6-year-old CPU. Thanks.

Edit: I also tried creating the following BuildConfiguration.xml (reference page) to see if this would resolve the issue. However, as you can see in the below logs, the number of processes did not increase and the compile time didn’t change (confirmed no time difference before and after).

<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
	<ParallelExecutor>
		<MaxProcessorCount>64</MaxProcessorCount>
		<ProcessorCountMultiplier>4.0</ProcessorCountMultiplier>
	</ParallelExecutor>
</Configuration>

UE5 Visual Studio build after adding the above xml:

1>Determining max actions to execute in parallel (4 physical cores, 8 logical cores)
1>  Requested 4 process count multiplier: limiting max parallel actions to 16
1>  Requested 1.5 GB free memory per action, 20.13 GB available: limiting max parallel actions to 13
1>Building 4 actions with 4 processes...
(...)
1>Total time in Parallel executor: 77.79 seconds
1>Total execution time: 79.53 seconds

Also, if I turn all the ParallelExecutor logs on in BuildConfiguration.xml, the following is evident in regards to the project module compilation. Not sure if “Wall Time” indicates anything meaningful:

1>Compile Module.proj.cpp [ Wall Time 79.54 s / CPU Time 93.14 s ]

(the other 3 actions each take 0-2 seconds each)

(note: also no change or improvement when setting Configuration::BuildConfiguration::MaxParallelActions to a high number like 12 or 24 in BuildConfiguration.xml)

Edit: Issue confirmed to still occur as originally reported on latest version, UE5.0.1 UE5.0.2 UE5.0.3

3 Likes
2

I noticed the same thing. Task Manager shows Utilization during UE4 compilations as pinned to 100% on all cores (though it still allows for interaction, luckily). UE5 compilation just results in partial usage of all cores. It’s as if they’re no longer supporting hyper threading.

Intel Core i7-10700K

1 Like
3

Good find! I’m glad it’s not just me.

I just checked Task Manager while compiling my UE5 project from Visual Studio and my CPU usage stays in the 20%-40%ish range the whole time.

I wonder if the number of processes is a red herring (even though the number is different when comparing UE4 and UE5), and there’s an unrelated factor which is limiting CPU usage and thus slowing down compilation.

4

I think limiting the processes to the number of physical cores certainly influences things. When it has a thread for each logical core, it can at least continue doing work when another thread is waiting for data.

1 Like
5

Yeah, either way, I hope there’s either a workaround or Epic is aware of this issue and patches it soon.

6

Hey there! I am getting the exact same problem. Did you find a solution or it’s still the same?
Edit: This solved my problem.

<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
    <ProjectFileGenerator>
        <Format>VisualStudio2019</Format>
    </ProjectFileGenerator>
        <BuildConfiguration>
            <MaxParallelActions>12</MaxParallelActions>
        </BuildConfiguration>
</Configuration>
3 Likes
7

I just double-checked using the BuildConfiguration.xml settings that you posted, but unfortunately I am still experiencing the same issues as in the original post. (low number of processes used, low CPU usage, total compile time more than double compared to UE4. Tested on the latest version, UE5.0.1 UE5.0.3).

8

Does anyone know if there is an official ticket for this issue? (or if this is a known issue by Epic?)
This thread’s been up for 3 months so I would hope that’s enough, but I can try submitting a ticket if that seems like a good idea.

Hoping this gets fixed in 5.1.0 …

1 Like
9

Relevant Post:

2 Likes
10

The key part of that post for me was the memory limit. I suppose there are multiple possible bottlenecks. And most were focussed on the parallelactions. It does seem to detect the number of cores and threads correctly, so I didn’t see a need to mess with that.
BuildConfiguration>MaxParallelActions can be set to 0 to disable this limit
ParallelExecutor>MemoryPerActionBytes can be set to 0 to disable this limit

Removing the memory limit risks the compiler running out of memory causing compilation to fail, which… surprises me, because I thought virtual memory was a thing, but apparently it really needs physical memory. And when Visual Studio needs several gigabytes just for existing, there’s not much left for the compilation.

It is not yet clear to me why UE4 didn’t have this problem.

For my situation I’ve settled on a variation on the configurations listed in the thread mentioned by mpxc, where the memory limit is set to avoid the heap memory errors of setting it to 0, and a limit to the number of max parallel actions to maintain some system interactivity.

<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
	<BuildConfiguration>
		<MaxParallelActions>8</MaxParallelActions>
	</BuildConfiguration>
	<ParallelExecutor>		
		<MemoryPerActionBytes>512000000</MemoryPerActionBytes>
	</ParallelExecutor>	
</Configuration>
1 Like
11

So, one thing I noticed is that my “logical cores” where not being used. RC-1290’s pointer to the BuildConfiguration settings led me to bAllCores. Now with this setting my computer is roaring.

<?xml version="1.0" encoding="utf-8" ?> 24 true 512000000
12

Some more FYI, using Incredibuild locally can also fix this issue, and was why most of our internal users did not encounter this issue in the first place.

13

I just upgraded from 5.0.3 to 5.1.1 and have been noticing significantly faster default compilation behavior without making any changes to the build configuration. Not sure if this is related to the engine upgrade, or an environmental factor, but in any case compilation times have reached an acceptable threshold on my end so I’m going to mark the question as answered. If anyone else is still having slow compilation times, feel free to open a new thread.

On a side note, I tested the following BuildConfiguration.xml configuration (%AppData%/Roaming/Unreal Engine/UnrealBuildTool/BuildConfiguration.xml) with bAllCores=true and noticed that the number of processes being used in compilation increased from 4 to 8 when doing a full solution rebuild (though remaining 4 when only modifying a single cpp file). I tried full-rebuilding couple times to compare this custom config to the default, and although the time range was pretty wide, there seems to be an improvement somewhere in the range of 5% to 20%, assuming this wasn’t a fluke. In any case, it’s probably worth trying out.

<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">

	<BuildConfiguration>
		<MaxParallelActions>24</MaxParallelActions>
		<bAllCores>true</bAllCores>
	</BuildConfiguration>
	
	<ParallelExecutor>		
		<MemoryPerActionBytes>512000000</MemoryPerActionBytes>
	</ParallelExecutor>	
	
</Configuration>
1 Like
14

Hello,

This starting to drive me crazy!

I migrated my project from UE4.27 to UE5.1, and now when I try to modify just a c++ line of my project, I get a huge compiling time.

Determining max actions to execute in parallel (18 physical cores, 36 logical cores)
Executing up to 36 processes, one per logical core
Requested max 24 action(s): limiting max parallel actions to 24
Building 1 action with 1 process…
…
Total time in Parallel executor: 89.15 seconds
Total execution time: 89.86 seconds

What I want to do is set my BuildConfiguration.xml to obtain:
(Building 1 action with 36 processes.)
exactly like UE4 did.

Could you help me with that?

I tried every solutions above but nothing seems to work.

Thank you,

Simon,

1 Like
15

I’m in the same boat here. I’d love to hear from anyone with any clue on a fix for this because all of the above solutions (even changing BuildConfiguration.xml) have made no difference. Maybe it’d be best to start a new thread?

Edit: Small update from looking through the UnrealBuildTool code.

  1. Using the xml tag MaxParallelActions in BuildConfiguration namespace will override the tags MaxProcessorCount and ProcessorCountMultiplier>in the ParallelExecutor namespace.
  2. The "Building 1 action with 1 process " message does not seem to indicate that in UE4 32 processes were actually being used to build that 1 action. Really the message we see now seems to better reflect what was going on before.

It seems like it is not the case that hyperthreading has been disabled, but that something is making the compilation itself take longer. It seems like it may be related to changes that were made to ManagedProcess.cs or the TaskGraph structure that manages the build processes.

I’m getting a little out of my depth, but build times are so bad that I’m gonna keep digging.