Using Clang to compile on Windows

S-ed · March 3, 2016, 5:04pm

Same question here. Wouldn’t it be better to maintain the code with just Clang for Windows and Linux?

Mattiwatti · June 17, 2016, 4:09pm

Hi folks,

Just bumping this thread for people who might be interested. I’ve submitted a PR on github to improve Clang support on Windows, including a basic output formatter which will highlight errors and warnings and make them clickable so you can navigate to the source of the error. Other than that, a fair amount of compiler errors have been fixed.

I really hope Epic will pick it up and start working on Clang support for Windows again (or ignore it and do it themselves, that is fine too). But if not, feel free to check out my branch from the PR and hack on Clang support yourself. I do recommend keeping a debug version of the latest Clang from trunk around, as it’s more likely than not that you’ll run into some compiler crashes

RCaloca · June 17, 2016, 8:23pm

@Mattiwatti: Thanks for reporting those warnings, got’em fixed now

Mattiwatti · June 18, 2016, 2:28am

Hehe, no problem! If that was your code you got unlucky, there were plenty of warnings to choose from

kamrann · June 18, 2016, 8:43pm

I agree it would be great if progress was made on this front.

Something that is really confusing me though - when I submit code plugins to the marketplace, they appear to be being compiled on Epic’s side using Clang. I’m submitting plugins that build without issue in MSVC, only to receive a list of errors back from Epic that look like Clang errors, which I can’t reproduce.

Is Clang now stable for building the engine? If so, it would be great if we could get some up-to-date information on setting it up. I tried building 4.11 from source with Clang not long ago and failed, as detailed here.

Mattiwatti · June 19, 2016, 6:04am

That is indeed strange. I don’t have any plugins on the marketplace, so this is the first I’ve heard of this. Clang definitely can’t compile UE on Windows at the moment, ~~and I don’t think it ever has been able to in the past~~ [edit: this is incorrect, see DarkVeil’s post from 01-05-2015 above - around UE 4.7 it was possible to compile in monolithic debug mode with Clang 3.7]. However it can compile many free-standing modules (plugins and game modules, and some standalone programs like ShaderCompileWorker) without issue, so long as the dependencies you’re linking against have already been built with MSVC. In fact I compile all of my plugins this way, because getting warnings from two compilers is always better than one (especially since Clang’s diagnostics are so good). So it’s certainly possible that Epic are using Clang to compile marketplace plugins. Why they would do that though I don’t know, given that most developers will be testing on VC++, and there are definitely compatibility issues between the two. It’s possible that the existing Marketplace was already using a Linux based server infrastructure, and that it was easier to set up an automated plugin crosscompilation toolchain using Clang on Linux than to move large parts of the backend system to Windows. This is just speculation though, I really have no idea.

The answer to that is a resounding no :D. You will most likely not be able to compile a single program with Clang on current releases (4.11/4.12/master), not even UnrealHeaderTool. There are a few reasons for this:

ICEs (i.e. compiler crashes, not errors). These are almost unavoidable when adding support for a new compiler to a project as huge as UE4, and they really really suck to debug. So far in working on this I’ve found three separate unrelated compiler crashes, one of which I’ve managed to identify as a compiler bug and filed a bug report for, but the other two remain a mystery and I’ve temporarily hacked around them for now.
UnrealdBuildTool isn’t up to date with current versions of Clang. This thread was posted in 2014 when the majority of Clang support for Windows was added, but there have been few (no?) updates to the toolchain since then. Meanwhile the constant stream of commits to UBT and other build related parts of the engine has continued, causing the two toolchains to diverge. As a result UE4 is now probably less useable with Clang on Windows than it was a year ago. Your linked thread is a good example of this; at some point someone added [FONT=Lucida Console]Arguments.Append(" /D_CRT_STDIO_LEGACY_WIDE_SPECIFIERS=1"); to VisualToolChain, but passing a preprocessor definition as an argument to Clang in this way is an error due to the different preprocessor specifiers used (/D vs -D).
Basic code incompatibility issues (again, introduced slowly over time due to nobody testing new builds with the Clang toolchain, but altogether this adds up after a while). While all of the major compilers generally have good standards support nowadays, C++ is so enormously complex that all compiler implementations are basically guaranteed to have incompatibilities somewhere. Fortunately these kinds of errors are usually straightforward to fix or work around (but there are a lot of them). Most of the compatibility problems I’ve come across tend to have to do with the dllexport attribute, static functions defined in a header file or template functions (in all three cases, LLVM/Clang is usually less eager than Microsoft to export symbol declarations).

It’s not all bad though; my PR has been accepted and should land in master some time next week! That should fix the biggest issues with UnrealBuildTool and Clang, as well as some compiler errors. With the patches from the PR applied (link here, or link to the repository here if you want to clone it), I am now able to compile every project in the UE4 solution with the exception of UE4Editor (obviously), CEF3 (some minor header issues that I haven’t fixed yet) and UnrealLightmass (includes an SSE header that is a 95% duplicate of Runtime/Core/Public/UnrealMathSSE.h, but just different enough that simply not including it won’t work). The UE4Editor project was at ~1500/1700 successful compilations last time I ran it, so not too far off.

kamrann · June 19, 2016, 4:18pm

Thanks for all the info @Mattiwatti, really appreciate it.

Yeah, I suspect with the marketplace thing, part of the reason may be that they want to harness the stricter compilation settings as a way of improving submitted code quality. Presumably after passing testing, the plugin’s Windows binaries get rebuilt with MSVC for release.
This seems perfectly sensible, aside from the fact that currently there’s no info on how we can replicate Epic’s build setup, so trying to get all errors and warnings cleared up is unsurprisingly a bit of a nightmare.

MS Codegen would seem to be an ideal compromise until UE4 fully supports Clang, but I think that would also require some changes to UBT.

ICEs are not fun. When I’ve encountered them with MSVC usually a bit of refactoring and simplifying of the code in the offending area has resolved the issue, but once in a while it can be much more of a pain to work around. I imagine they may still come up somewhat more frequently with Clang.

Thanks for your great work on this, I hope @DarkVeil can get back on it before long.

Zlecawiigo · January 15, 2017, 2:07pm

Any updates to this?

Nightmask3 · January 23, 2017, 11:01am

Man you are fighting the good fight. I could not even imagine how I would deal with a compiler bug.

I begin to understand why some people reject C++ for C now.

havu_official · September 7, 2018, 3:55pm

Any updates to this?

TheDreamCatcher · September 11, 2018, 6:56pm

I would also love to hear more about the transition. I know UE4 i sshipping clang-5 on Linux (which on most recent distros looks like a step back from clang 6) but I’d love to have consistancy on all platform about it.

havu_official · September 17, 2018, 7:37am

most of platforms are compiling with clang. but only Windows does not use clang.
when i try build ShooterGame(sample game from marketplace) with clang on Windows, it occurs syntax error, despite actually not syntax error.
I mean, it compiles well on MSVC even not touch any codes. besides, compiles well other platforms that using clang.

** possibilities **

differences of “include path” between other platforms that use clang.
several potential bugs on UE4
Invalid setting on UnrealBuildTool’s clang support routine(bCompileWithClang)

if you guys have any ideas, please share here.

I’ll share if I have a good idea.

anonymous_user_87ccf054 · September 18, 2018, 10:12am

So tried to compile UE 4.20 and clang 6

19 succeeded, 16 failed, 10 up-to-date, 7 skipped
It appears that allways the same errors occur:

Error conflicting types for

InitializeSRWLock
ReleaseSRWLockExclusive
ReleaseSRWLockShared

havu_official · September 27, 2018, 6:55am

tried UE 4.16 and clang 6

include path is 99% same. (except MSVC libraries)

Paul_TBBle_Hampson · November 30, 2018, 12:56pm

I decided to give this a go, and I have the UE4 Editor building and starting. I haven’t tested beyond that yet though.

I pushed my branch to https://github.com/TBBle/UnrealEngin…ster-clang-wip based on https://github.com/EpicGames/UnrealEngine/tree/master, preparing to submit a pull request. I also have an older version at https://github.com/TBBle/UnrealEngin…ease-clang-wip based on https://github.com/EpicGames/UnrealEngine/tree/release (Currently 4.21.1).

I’ve put up a pull-request for Epic at https://github.com/EpicGames/UnrealEngine/pull/5376 since it’s generally compiling (barring the below) and hence usable for my use-case.

Caveats:

I will rebase my branch, as I either find new fixes, Epic’s ‘release’ or ‘master’ branches move, or because I feel like it.
I killed almost all the warnings, except some coming from Intel Thread Building Blocks (Used by ProxyLODPlugin) calling the Deprecated [FONT=courier new]_ReadWriteBarrier](_ReadWriteBarrier | Microsoft Learn) function.
I had to pull in an update to Boost for the ProxyLODPlugin, and I didn’t trim it at all. It’s header-only, but probably includes headers that we don’t need, or that depend on compiled libraries.
I built it with Clang/LLVM 7.0.0. Linux uses Clang/LLVM 6.0.1 for 4.21, so I may have changes that don’t compile with the older Clang.
No idea what OS X uses, and there’s no cross-compile setup for it, so I’m just going on hope there.
Debug Editor config fails to build, I’ve only really tested Development Editor. (See first edit for failure details)
Test config fails, I suspect it’s actually a code-fault when the [FONT=courier new]STATS macro is disabled.

To build with Clang, I used the LLVM 7.0.0 “Windows (64-bit)” from LLVM Download Page. The compiler setup is identified by UBT as:

To enable Clang, in [FONT=courier new]Engine\Saved\UnrealBuildTool there is a file BuildConfiguration.xml. Mine looks like:


<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
    <WindowsPlatform>
        <!-- Build using Clang instead of Visual Studio 2017 -->
        <Compiler>Clang</Compiler>
    </WindowsPlatform>
</Configuration>

My actual goal here was to move towards generating a [FONT=courier new]compile_commands.json for UE4, (like Get compile_commands.json - Platform & Builds - Epic Developer Community Forums), so that I could apply clang-tooling-based tools to the UE4 code-base, and perhaps even take advantage of Visual Studio Code’s C++ Extension’s Intellisense Engine, since Visual Studio 2017 is a somewhat heavy IDE for my poor laptop. [HR][/HR]
Edit: Debug build fails to link, I suspect this one’s a clang fault. The symbol is [FONT=courier new]__forceinline and is also a non-virtual inline member function, so clang didn’t emit a version of it when generating the Render module’s DLL, as all callers should be inlining it. However, when calling it, clang has emitted a call to the version exported from the Render module’s DLL instead of inling the code.



9>Module.Engine.23_of_41.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: void __cdecl FMeshMaterialShader::ValidateAfterBind(void)" (__imp_?ValidateAfterBind@FMeshMaterialShader@@QEAAXXZ) referenced in function "public: class FShader * __cdecl FMeshMaterialShaderType::FinishCompileShader(class FUniformExpressionSet const &,class FSHAHash const &,class FShaderCompileJob const &,class FShaderPipelineType const *,class FString const &)" (?FinishCompileShader@FMeshMaterialShaderType@@QEAAPEAVFShader@@AEBVFUniformExpressionSet@@AEBVFSHAHash@@AEBVFShaderCompileJob@@PEBVFShaderPipelineType@@AEBVFString@@@Z)
9>D:\Unreal\UnrealEngine\Engine\Binaries\Win64\UE4Editor-Engine-Win64-Debug.dll : fatal error LNK1120: 1 unresolved externals

Paul_TBBle_Hampson · January 4, 2019, 2:29am

The pull request was accepted and has been merged into [FONT=courier new]DEV-BUILD. My branches on github that I mentioned earlier will go away some time after Clang-CL support lands on the [FONT=courier new]master](https://github.com/EpicGames/UnrealEngine/commits/master) branch. It sounds like there’s now going to be Clang builds in the Epic CI system, so I expect it’ll remain functional for future releases.

Thanks to [USER=“1345”]Ben Marsh[/USER] on the Epic side for merging my changes.

Megakoresh · May 11, 2019, 6:06am

I think this is really valuable addition to the engine, as it allows the use of the same compiler and code analysis tools regardless of the OS, not to mention that clang is just a great tool overall and has allows for much better experience with VSCode, which in turn provides a much better development experience than Visual Studio. I hope your changes make it to release soon, it’d be fantastic to switch to clang and use vscode without having to do a bunch of hacks to compile projects.

anonymous_user_2369a74d · May 13, 2019, 7:36am

[USER=“1210893”][/USER], how far did you get with generating a compile_commands.json file? Is this something you managed to get working?

jeevcat · May 13, 2019, 7:41am

Paul “TBBle” Hampson, how far did you get with generating a compile_commands.json file?

Paul_TBBle_Hampson · May 21, 2019, 9:07am

They should be in 4.22, but I’m on a 4.21.2 project at the moment, so haven’t had a chance to try them out. Annoyingly, 4.22 also contains a Visual Studio Code fix which was broken in 4.21, so I’m currently fully on the MS Visual Studio flow right now.
[HR][/HR]

I spent a bit of time poking at UBT, trying to trace-down how it generates Intellisense preloading. As it turns-out, it does an entire build but with an [FONT=courier new]isIntellisense flag plumbed all the way through that causes the compilations to not run, and then outputs the Intellisense data.

I’d prefer a more-generic approach, but UBT’s not exactly the easiest code-base to parse, in a generic sense. Lots of platform-specific stuff, that pokes through to fairly high layers.

I haven’t really had the time to sink more into that direction. If I do, I’d post it as an answer at Get compile_commands.json - Platform & Builds - Epic Developer Community Forums.

Coincidentally, I did write a compile_commands.json extractor for MSBuild, but of course UE doesn’t use MSBuild to build, so it doesn’t help for this work. Also, written on work time so not published open-source, although I’d like to.