Emacs, Clangd and Unreal Engine

[drcxd]

I am an emacs user and am trying to get emacs and clangd work with Unreal Engine Code. I have used UBT to generate the compile-database with the argument -mode=GenerateClangDatabase.

I run emacs and clangd in Windows Subsystem for Linux (WSL). Though clangd could parse the compile-database file correctly, it seems that it still need something else to compile the code. In the log of clangd I could find something like this:

I[22:35:10.483] Indexed /mnt/d/workplace/UnrealEngine/Engine/Source/Runtime/RenderCore/Private/RayTracingInstanceCopyShader.cpp (23607 symbols, 77274 refs, 469 files)
I[22:35:10.490] Failed to compile /mnt/d/workplace/UnrealEngine/Engine/Source/Runtime/RenderCore/Private/RayTracingInstanceCopyShader.cpp, index may be incomplete

As a result, I cannot take advantage of the full functionality clangd provides. It can’t find definitions of some symbol. To be more specific, clangd could not find definitions for most of the symbols in a header file. It works for symbols in some simple structs at the beginning of the header. However, it failed to find definitions for everything else in the file. For example, in the file PlayerCameraManager.h, clangd could find definitions for the struct FMinimalViewInfo, but it could not find definition for AActor which appears as the base class of APlayerCameraManager.

I am not sure this is a problem of the compile-database file which is produced by UBT, or it is a problem of clangd. Since we can already compile Unreal Engine using Clang, I suppose clangd could also parse Unreal Engine source code. I hope someone in the Unreal community could help me with that since I have no knowledge about programming language, compiler and Language Server Protocol (LSP), but I really want to combine these powerful tools: emacs, LSP, clangd and Unreal Engine.

[drcxd]

I dig into the code and found more useful information.

Include Directories

First the command I used to generate the compile database is

 UnrealBuildTool.exe -mode=GenerateClangDatabase -project="YourProject.uproject" -game -engine ProjectName Development Win64

Then I checked some of the error messages clangd prompted. First, it said it can not find the include file sal.h. After some simple research I found that this file is a Windows exclusive file and does not exist in WSL. This is reasonable since the command specified that the target platform is Win64. I came up two methods to solve this problem.

Switch to Another Platform

First, I could change the UBT command to specify other platform such as Clang or Linux. Here is the output:

PS D:\workplace\UnrealEngine\Engine\Binaries\DotNET> .\UnrealBuildTool.exe -mode=GenerateClangDatabase -project="D:\workplace\LearnUnreal\LearnUnreal.uproject" -game -engine LearnUnreal Development Clang
ERROR: No platforms specified for target
PS D:\workplace\UnrealEngine\Engine\Binaries\DotNET> .\UnrealBuildTool.exe -mode=GenerateClangDatabase -project="D:\workplace\LearnUnreal\LearnUnreal.uproject" -game -engine LearnUnreal Development Linux
ERROR: Platform Linux is not a valid platform to build. Check that the SDK is installed properly.

I am not familiar with UBT and am still reading these documents, so for now I do not know how to proceed in this way.

Add Include Directories

The other method is to either configure UBT to include the Windows specific include directories in the output compile database or we add them to the compile database manually. Again, I do not know how to configure UBT to include those additional include directories. Although I could add them manually myself, it looks much like a hack not a formal solution.

Macro Substitution

Then I found that Cland has difficulties in expanding Unreal Engine macros. For example, UCLASS is expanded to

BODY_MACRO_COMBINE(CURRENT_FILE_ID,_,__LINE__,_PROLOG)

The macro is defined as

#if UE_BUILD_DOCS || defined(__INTELLISENSE__ )
#define UCLASS(...)
#else
#define UCLASS(...) BODY_MACRO_COMBINE(CURRENT_FILE_ID,_,__LINE__,_PROLOG)
#endif

Microsoft Visual Studio could expand this macro correctly because the flag __INTELLISENSE__ is defined. However, this is not true for the compile database generated for Clang. Again, we could manually add the definition to the command database, but I prefer more general solution.

Another macro is GENERATED_BODY, it is defined as:

#define GENERATED_BODY(...) BODY_MACRO_COMBINE(CURRENT_FILE_ID,_,__LINE__,_GENERATED_BODY);

Clangd complains this with C++ requires a type specifier for all declarations. I do not know how MSVS deals with this, but it is obvious these macros are preventing Clangd from compiling the code. Thus it can not correctly find definitions for all the symbols.

I hope there will be some one who can help with this.

[manenko]

Hi there!

You can try a project of mine which consists of two parts:

1. Unreal Editor plug-in adds Emacs to the “Source Code Editor” combo box and makes Emacs a default editor for a project source files. This means Unreal Editor will open Emacs instead of Xcode/VisualStudio/Whatever.
2. A patch for the UnrealBuildTool creates a Clang compilation database in the project’s root directory. The database could be used by lsp, rtags and any other Emacs autocompletion backend. You must have Emacs selected as a source code editor for the project to make it work. The Unreal Editor will invoke the plug-in’s magic each time it wants to refresh the project.

As I said, you have to set up an autocompletion backend on your own. The plug-in doesn’t do that for you. Personally, I use lsp. You can look into my Emacs configuration directory for an inspiration on how to configure it. The main points of interest, I believe, are:
init-company-mode.el
init-cpp.el
init-snippets.el

Beware, both the plug-in and patch “work on my computer” under macOS, but I believe they should work on GNU/Linux and Windows. The project has a README.md which explains how to install and use it.

Let me know if this helps.

[drcxd]

Thanks for your help. I will check that later. Actually, I found a way to make my setup work, but only for a while. I defined the __INTELLISENSE__ macro for clangd. This makes those UCLASS expand to nothing which finally makes clangd could parse the source code.

However, the UBT generated code is not included in the compilation database thus it is invisible to clangd. Unfortunately, there are some access modifiers generated in those .generated.h files which makes clangd “incorrectly” recognize the accessibility of some members. While, in the perspective of clangd, it works well, but it just not play nice with Unreal code under the current compilation database setting.

Further more, when clangd is compiling in the background, I found it frequently crashed when it encountered some particular file. I do not know which file makes it crash yet.

To some extend, I start believing that it is almost impossible to make clangd works with Unreal code, because Unreal code is NOT standard C++ and clangd is designed for standard C++.

Anyway, thanks for your reply. I will try your setup and see if it could work for me.

[manenko]

@drcxd

To some extend, I start believing that it is almost impossible to make clangd works with Unreal code, because Unreal code is NOT standard C++ and clangd is designed for standard C++.

Cheer up! I believe it is possible. There is no non-standard C++ once Unreal Header Tool is run on the project.

I also found that the existing compilation database generation in UBT uses “Intellisense” mode which doesn’t work well on macOS. What I do instead, I generate the database using the same environment UBT uses to build the project.

[drcxd]

Thanks, I some how make it work. Although I did not use your project directly, but I got a lot of inspiration from it. Basically, what I was trying to do was to compiling Unreal Code using a Linux version clang in WSL, which is difficult to achieve since it at least require I set up the cross-compiling for the project.

After I switch to the Windows native Emacs and Clangd, everything works fine. During this process, I also have learned more about how these macro works actually with the help of UHT.

However, there is still another problem bothering me. The thing is when I edit some content of a header file, to be specific, when I make some edit which changes the line number of the GENERATED_BODY macro, the header file is broken. This is due to the #define in the .generated.h file can’t match the GENERATED_BODY after the line number has changed. Before the next time I run the UHT, those error messages clangd send to me persist.

The only way I know how to run the UHT is through the IDE’s building process, which it automatically run UHT and generate the headers. If I could find out how to run UHT manually, then I can update my generated header files without building the project after I made some simple changes to my header file. Sometimes, those changes are too naive to issue another build, e.g., including another header file or forward declaring a class name, etc…

I have tried Jet Brain’s Rider for Unreal and it works that way, I mean, automatically running UHT in background to update generated header file to make everything in the same page.

Unreal document talks really little about how to execute UHT manually, if you know how to do that, please tell me, that would be a really great help.

P.S. Clangd is somehow slow to reflect changes in header files, even UHT is not involved.

[manenko]

@drcxd

The only way I know how to run the UHT is through the IDE’s building process, which it automatically run UHT and generate the headers. If I could find out how to run UHT manually, then I can update my generated header files without building the project after I made some simple changes to my header file.

Running UHT manually is complicated because UBT generates a special manifest file for UHT with build modules metadata and who-knows-what-else.

However, you can do this instead:

bash "/Users/Shared/Epic Games/UE_4.27/Engine/Build/BatchFiles/Mac/Build.sh" MyProjectNameEditor Mac DebugGame -SkipBuild -project=/Path/To/MyProjectName.uproject

The -SkipBuild flag instructs UBT to generate makefiles, run UHT if needed and stop. And this is fast

[Rawalanche]

Despite not being targeted exactly at Emacs, I think this video will give you most information you need:

Unreal Engine C++ Project Setup, From Scratch

I also can not recommend this channel enough:

Alex Forsythe

This guy requires way more attention than he gets. His videos are so amazing, high effort content.

[drcxd]

This helps a lot, now I can generate those .generated.h files without building my whole project every time I just make a tiny editing to my header files. Now I am pretty satisfied with my programming environment using Emacs, Clangd and Unreal Engine.

Thank you a lot!

[drcxd]

Thanks, I’ll take a look later.

[manenko]

You are welcome!

[OC_DChepuryshkin]

Hello there!
There is another option to feed clang custom arguments and include pathes that not added by default: you can add .clangd configureation file and tweak clangd inany way you want. Here’s the link to doc Configuration

P.S.
I see, that this topic is pretty much dead, but I think my advice might be useful for many vim/emacs enthusiasts

[ranjithshegde]

Are you personally using the .clangd file? If so, would you please mind sharing an example? I honestly have no idea what such a file should look like.

I am on a similar journey (nvim + clangd). There are plugins to parse and automate all the build tasks in .vscode directory. So I have all the build/debug/tasks portion covered.
Just cant get lsp to work properly.

[drcxd]

My .clangd file:

CompileFlags:
  Add: [-D__INTELLISENSE__, -ferror-limit=0]

Diagnostics:
  Suppress-Wc++17-extensions
  UnusedIncludes: Strict

Plese note that I am using Visual Studio to compile my Unreal project on Windows platform. The -D__INTELLISENSE__ is only meaningful in this context.

[drcxd]

@manenko Hi, UE5 has been out for a while and I see there are lots of changes on the building process. UHT is deprecated and its functionality is integrated into UBT. What I have learned about generating the compilation database and header files are also deprecated now. Do you have any idea how could we use the new UHT to generate the compilation database and header files now?
I tried the old command

UnrealBuildToool.exe -mode=GenerateClangDatabase -project=<path-to-uproject.file> <project-name> Development Win64

It did generate a “compile_commands.json” file but the content in it seems not correct.

Edit: The content in the compile_commands.json looks correct to me now. It’s like:

{
		"file": "D:\\workplace\\CitySample\\Plugins\\CitySampleMassCrowd\\Source\\CitySampleMassCrowd\\Private\\CitySampleDebugVisualization.cpp",
		"command": "\"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\bin\\HostX64\\x64\\cl.exe\" @D:\\workplace\\CitySample\\Plugins\\CitySampleMassCrowd\\Intermediate\\Build\\Win64\\UnrealEditor\\Development\\CitySampleMassCrowd\\CitySampleDebugVisualization.cpp.obj.response.gcd",
		"directory": "D:\\workplace\\UnrealEngine-release\\Engine\\Source"
	}

It includes another file by using the @ sign. The content in the included .gcd file is like

D:/workplace/CitySample/Plugins/CitySampleMassCrowd/Source/CitySampleMassCrowd/Private/CitySampleDebugVisualization.cpp
/I D:/workplace/UnrealEngine-release/Engine/Source
# more include directories...
/external:W0
# more other options...

Form the log of clangd, I found that only the part after the /external:W0 get passed to clangd.

"C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\bin\\Hostx64\\x64\\cl.exe" --driver-mode=cl /external:W0 /external:I "ThirdParty\\mimalloc\\include" /external:I "ThirdParty\\LibTiff\\Source\\Win64" /external:I "ThirdParty\\LibTiff\\Source" /external:I "ThirdParty\\Ogg\\libogg-1.2.2\\include" /external:I "Developer\\DistributedBuildInterface\\Public" /external:I "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\\VC\\Tools\\MSVC\\14.31.31103\\INCLUDE" /external:I "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\ucrt" /external:I "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\shared" /external:I "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\um" /external:I "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.18362.0\\winrt" /FID:/workplace/UnrealEngine-release/Engine/Intermediate/Build/Win64/UnrealEditor/Development/Engine/Definitions.h /sourceDependencies D:/workplace/UnrealEngine-release/Engine/Intermediate/Build/Win64/UnrealEditor/Development/Engine/Actor.cpp.dep.json /Zc:inline /nologo /Oi /FC /c /Gw /Gy /Zm1000 /wd4819 /D_CRT_STDIO_LEGACY_WIDE_SPECIFIERS=1 /D_SILENCE_STDEXT_HASH_DEPRECATION_WARNINGS=1 /D_WINDLL /D_DISABLE_EXTENDED_ALIGNED_STORAGE /source-charset:utf-8 /execution-charset:utf-8 /Ob2 /Ox /Ot /GF /errorReport:prompt /EHsc /DPLATFORM_EXCEPTIONS_DISABLED=0 /Z7 /MD /bigobj /fp:fast /Zo /Zp8 /we4456 /we4458 /we4459 /wd4463 /we4668 /wd4244 /wd4838 /GR- /W4 /std:c++17 -D__INTELLISENSE__ -ferror-limit=0 "-resource-dir=c:\\Program Files\\LLVM\\lib\\clang\\15.0.0" -- "d:\\workplace\\UnrealEngine-release\\Engine\\Source\\Runtime\\Engine\\Private\\Actor.cpp"

As a result, missing lots of include directories, clangd reports a lot of “file not found” error.

[drcxd]

Updating LLVM to version 15.0.6 fix this problem on my PC.

[Hectarea1996]

After a few months trying this out, finally I’ve made it! It works!

I want to share what I did so far, although there is still a bit of work to do. I’m using Windows 11, but I think this can be changed a bit in order to make it work in other patforms.

1. Install LLVM. You can do this from its GitHub page at the release section. I downloaded the LLVM-16.0.1-win64.exe installer. Select the default installation path.

2. Install Clangd from its GitHub page at the release section. I downloaded the clangd-windows-15.0.6.zip. You place the content wherever you want. I placed it at ‘C:\clangd’, for example.

3. To enable jump definitions and autocompletions you need to generate the compile_commands.json file. Just execute the next command:

PathToUnrealEngine\Engine\Build\BatchFiles\Build.bat -mode=GenerateClangDatabase -project="PathToYourProject\ProjectName.uproject" ProjectNameEditor Development Win64

Note here the ‘Editor’ suffix after ProjectName. This is really important.

The compile_commands.json is generated at the UnrealEngine’s root directory. Just copy and paste it in your project’s root directory.

4. To enable correct linter with flycheck you need clang-tidy. It comes with LLVM, but you need to tell flycheck to use it. Just install the flycheck-clang-tidy Emacs package.

Just a little problem here that @drcxd stated: Every time you change a bit your header, thinks are broken. Thankfully, @manenko said the solution just below. Just do this:

a. Run the following command:

PathToUnrealEngine\Engine\Build\BatchFiles\Build.bat ProjectNameEditor Win64 DebugGame -SkipBuild -project"PathToYourProject\ProjectName.uproject"

b. Run inside Emacs the command flycheck-buffer.

The idea here is to execute these two commands each time you change the file or every 5 seconds or every time you save the file.

[Hectarea1996]

On the Emacs side we need to do some configurations. I’m leaving at the end a link to my emacs config.

We need the following packages to get autocompletions and a good linter: lsp-mode, flycheck-mode, flycheck-clang-tidy and company-mode.

Also, you need to tell lsp where the clangd executable is. You need to setq the lsp-clangd-binary-path variable.

Optionally, you should use these packages too: yasnippet, ue-mode (part of the @manenko 's project), lsp-ui and lsp-ivy.

Lastly, some notes about ue-mode and linter. Remember the problem @drcxd stated? I wrote a function that calls the Unreal command and flycheck-buffer every time you save a file. Additionally, I made convenience commands to compile and run a project. These commands are placed just before the ue-mode configuration in my init file. Also, ue-mode is not finished yet but @manenko can’t continue implementing the mode right now. I needed to change some functions to make it work the way I wanted. All of these changes are in my repository. Feel free to investigate and copy all the changes I made.

Here is my emacs config repository: emacs-windows
Here is the init file: init.el
And here is the modified ue-mode directory (this path could break soon) : ue-mode directory

Also, I made some additional Unreal Engine snippets placed at the previous directory. Feel free to copy them if you want. However, there are still some that have been written by @manenko (just an advise for copyright reasons).

Hope this helps to make easier the linkage between Emacs and Unreal Engine. Happy hacking! (I’ve always wanted to say that)

[DGhandehariMant]

Hi, I’m in Windows trying to run clangd (15.0.6) instead of Intellisense. In my case, I’m using VSCode. But, we are building with MSVC. I have something that works OK, but there are a couple of things that aren’t perfect.

I think the biggest issue is precompiled headers. We have several modules, all of which have different precompiled header sets. The PCH is then auto-included in all the files via the command line. clang has its own precompiled headers, but we are not actually running clang for the build, so they don’t get generated. Additionally, the UBT-generated compile_commands.json makes no mention of including them.

Anyone encountered this, or have ideas on how to fix it?

[lukas.vilim]

I see a lot of you copying around the compile_commands.json that the UBT generates in the engine root. There is absolutely no need to do that. Just add -OutpuDir=/desired/directory/.