Please clean up the engine includes

I realize this is a very big job, and if it was considered a priority would have been tackled a long time ago, but I’m going to bring it up anyway. It causes me hassle every day and I’ve no doubt it does to most anyone writing C++ with UE4, whether or not they’re aware of the underlying issue.

So to be blunt, the engine includes are a total mess. Specifically, I’d say there are two primary problems - overreliance on precompiled headers, and lack of organization of public headers.

Precompiled headers
UBT’s enforcement of putting the PCH include before all others leads to the problem of engine headers reliant on includes within the PCH in order to compile. This is rife throughout the engine, to the point where there are many headers that don’t bother to include any dependencies whatsoever, but just assume the declarations they rely on have been included via the module’s PCH.

The result is bizarre compilation errors deep inside engine code when using an engine header in a module that doesn’t happen to have all the same includes within its PCH as the module in which the engine header was developed. This leads to trawling through the API reference/code to find in what header the offending type is declared, then working out whether that header should be included directly or indirectly via some other, etc. Not only does this happen during active development, but also regularly when a new engine version is released, due to modifications made to engine headers/PCHs.

Public headers
Given the size of the engine, clearly it’s not reasonable to specify low level individual includes for every single thing needed. That said, without a well organized and documented set of ‘batch include’ headers that just wrap includes to related types, the fallback in practice is including high-level headers that end up pulling in half the engine via nested includes. I can’t imagine this is good for compile times even with PCH use.

Furthermore, it means the PCH files themselves end up huge. A PCH that includes only ‘Engine.h’ comes out at around 1GB. With plugins with multiple modules becoming more and more widely used, disk usage is becoming a big problem because of this. Simply disabling PCH generation doesn’t help as the object files all end up much bigger instead. The fundamental problem is that most code has no use for 95% of what gets included via ‘Engine.h’, so most of the disk usage is wasted.

What could help
I don’t know if perhaps the PCH having to come first is based on some important reason relating to compilation times. Assuming it’s something that can’t really be changed, a preprocessor define or UBT flag that suppressed PCH use could at least aid in verifying self-contained headers. Engine modules could periodically be built with this enabled to catch occurrences of PCH-reliant headers. Obviously, removing all the existing such cases would be a substantial job.

As for the second issue, I think a refactor of public headers is needed. Ideally there would be a set of documented headers for external use that consisted purely of other include statements. For smaller modules there could just be a single module header, but generally it would need breaking down into individual headers for logically related sets of functionality. This would aid both in cutting down the amount of code included by our own headers, as well as in controlling what went into our module PCH files so as to save on disk use.

I would like to second this, particularly the difficulties with “…bizarre compilation errors deep inside engine code when using an engine header in a module that doesn’t happen to have all the same includes within its PCH…”. I realize it’d be a serious undertaking, but if this kind of thing could be done for all new modules and code added, and then over time worked through the code-base, it’d be a start! Having to just blindly include Engine.h to fix lots of issues is crazy.

Yes please, here comes a bad example of some includes of my MaterialManager:

I’m also forced by the engine to use the same name for the precompiled header
in my library files which are shared between multiple projects. So I can’t use the
project name, but instead use “precompiled.h”.

Which at some point in time will lead to big misunderstandings of the compiler,
especially concerning plugins.

#include "precompiled.h"
#include "MaterialManager.h"
#include "MaterialNode.h"
#include <iostream>
#include "UnrealEd.h"
#include "Materials/MaterialExpressionClamp.h"
#include "Materials/MaterialExpressionComponentMask.h"
#include "Materials/MaterialExpressionConstant.h"
#include "Materials/MaterialExpressionConstant3Vector.h"
#include "Materials/MaterialExpressionLinearInterpolate.h"
#include "Materials/MaterialExpressionPower.h"
#include "Materials/MaterialExpressionMultiply.h"
#include "Materials/MaterialExpressionSpeedTree.h"
#include "Materials/MaterialExpressionTextureCoordinate.h"
#include "Materials/MaterialExpressionTextureSample.h"
#include "Materials/MaterialExpressionTwoSidedSign.h"
#include "Materials/MaterialExpressionVertexColor.h"
#include "Materials/MaterialExpressionMaterialFunctionCall.h"
#include "Materials/MaterialExpressionCustom.h"
#include "Materials/MaterialExpressionTime.h"
#include "Materials/MaterialFunction.h"

Hello! I can confirm that we are doing a lot of work to fix up the includes for a future release. It’s a big undertaking, but when we’re done it should be a fraction of the amount of text on any given list of includes. Once we have something more solid to say on the update, we’ll share more info.

Hey, seems it was finished today!

Yes please! :slight_smile:

I would like the tutorial on this topic when it launches :confused:

This looks awesome. I should request things more often. :wink:

My small suggestion about resolving issue with precombiled headers.

Change “NamePCH.h” to “stdafx.h” (its target solution for VisualStudio) or make special parser enforcing like a #include precompiled and UHT will be expand it in properly name

Nice one :slight_smile: