Error building Unreal Engine 4.26 on Linux Debian 11 (Bullseye)

Development Platform & Builds
, , , ,
21

UPDATE: By some miracle I finally got UE4 v4.26.1 to successfully build in Ubuntu 18.04.

I’ll try my best to convey what I did because I am still not exactly sure what it was that fixed it. I think it might have been the process I followed, so maybe context is key.

**Context**
I noticed during my last build (related to my previous post) the OS was telling me that I was running low on disk space. I checked and I still had 64 GB free. I looked at top and saw that there was ~20 GB of swap being used (I didn't partition a swap space when I setup this system). I guess Ubuntu carves out space on its own. I used gparted to expand my SSD space by another 40 GB. Obviously this required that I reboot, which might have been helpful to release any file locks. I have 32 GB RAM but not all of it was being used during the build (~12 GB used). Michael Sartain asked about RAM a couple of times and I think that was important. When I went to build UE4 the last time, the only makefile that was generated was for UE4Editor. All previous attempts built CrashReportClient, CrashReportClientEditor, ShaderCompileWorker, UnrealLightMass, UnrealFrontEnd, and then UE4Editor (in that order consecutively). I am thinking maybe that by building the UE4Editor on its own, gave me enough free disk space/memory to not get those **clang++: error: clang frontend command failed due to signal (use -v to see invocation)** errors on random modules.

Anyways, here are the simple commands I entered after my long journey in desparation.

from within the root UnrealEngine git folder:
`$ git pull`
`$ git clean -d -fx -i`
`$ ./Setup.sh`
`$ ./GenerateProjectFiles.sh`
`$ make -j$((`nproc`-1))
`$ make ShaderCompileWorder` (this wasn't available when I tried to start UE4Editor)

Good luck to anyone that experiences the temporary suffering this has caused/causing :).

22

glad to hear that you solved! Unfortunately in my case I have a lot of free disk space and RAM… I’ll try to compile one program at a time…

23

Can you try doing the following :

  1. Run ./Setup.sh && ./GenerateProjectFiles.sh
  2. modify your BuildConfiguration.xml to set the max processor count (like below)
  3. Run “ARGS=-clean make” and then make, and see if this fixes your issue?

Sorry - just trying to throw some things at the wall and see if we can narrow down what’s going on. Thanks!

mikesart@mikesart-tr3:/epic/p4/UE4/UE4.git (4.26)$ cat Engine/Saved/UnrealBuildTool/BuildConfiguration.xml
<?xml version="1.0" encoding="utf-8" ?>
<Configuration xmlns="https://www.unrealengine.com/BuildConfiguration">
        <ParallelExecutor>
                <MaxProcessorCount>4</MaxProcessorCount>
        </ParallelExecutor>
        <LocalExecutor>
                <MaxProcessorCount>4</MaxProcessorCount>
        </LocalExecutor>
</Configuration>
24

Ok, this is what I did:

  1. installed fresh new SSD in my PC
  2. formatted ext4
  3. git clone -b release on new SSD
  4. ./Setup.sh —> OK
  5. ./GenerateProjectFiles.sh —> OK
  6. make —> same old errors
  7. git clean -d -fx -i
  8. ./Setup.sh —> OK
  9. ./GenerateProjectFiles.sh —> OK
  10. edited build conf as Michael suggested
  11. make…

this time I noticed that first programs compiled without errors until “UnrealFrontend” gave:

malloc(): invalid next size (unsorted)
Stack dump:
0.      Program arguments: /media/ssd/UE426.git/Engine/Extras/ThirdPartyNotUE/SDKs/HostLinux/Linux_x64/v17_clang-10.0.1-centos7/x86_64-unknown-linux-gnu/bin/clang++ @/media/ssd/UE426.git/Engine/Intermediate/Build/Linux/B4D820EA/UnrealFrontend/Development/ProjectLauncher/Module.ProjectLauncher.cpp.o.rsp
1.      <eof> parser at end of file
2.      Per-file LLVM IR generation
malloc(): invalid next size (unsorted)

So… maybe a little better? Anyway not solved :frowning:

25

Maybe try running memtest on this machine? Otherwise I guess my only idea would be to try something like:

strace -o /tmp/ue4make -T -ff – make

And look to see if there are failures in the massive number of log files it’ll have?

Sorry… wish I had better suggestions but I’ve literally built UE4.26 on my Debian Bullseye machine hundreds of times and I haven’t been able to reproduce this at all, unfortunately.

26

Thank you Michael for your time and your support, I’ve tried with BuildConfiguration.xml modified as you suggested and then:

make ARGS=-crean

and:

make -j1

Surprisingly I had the first error a lot after the usual (or maybe it took a long time to get there).
After that I run “make -j1” again. And again after the second error.
After third make command it completed compilation and now I’m able to run UE 4.26…
It seems that each time I run make it is able to compile previously error files and proceed.
It doesn’t make sense to me but at least I can run 4.26! It seems not very stable but for simple projects works well.
I suppose that I will settle with this situation until a new version comes out, or in a few months when the current Debian Testing becomes Stable, I could take the opportunity to clean install, in these years it would not be the first time that Testing plays strange tricks on me .

27

Yeah, I’ve been running Debian Testing for years and never had any issues like this. Not sure what’s going on there. I’ll let you know if I can ever get this to repro and also maybe, hopefully a clean install will fix things for you. Thanks .

28

Hi, just a small update on this issue…

Debian 11 Bullseye has become Stable this summer, I’ve upgraded my system to last stable version but nothing has changed and I have the same problems as written above.

I thought it was caused by a messy system since I managed to get PC to work with Testing release, so I formatted my disk and made a fresh new installation of Debian 11 Bullseye.

Once the system was ready I’ve cloned Unreal git repo to a new folder and ./Setup.sh && ./GenerateProjectFiles.sh && make… and got the same errors :frowning:

I’ve gone through all the log and I’ve found 2 errors/issues:

1. error extracting clang package (not detected)

when Setup.sh installs the dependencies seems to not check the correct extraction for v17_clang tar.gz package. I can see an error log extracting files from this archive but Setup script end with SUCCESS anyway…

I’ve downloaded and extracted manually this archive, but this does not fix my problems.

2. problem with number of physical core detection and multithreading

When compiling modules, I can see this:

logical and physical cores.jpg
logical and physical cores.jpg854×92 28 KB

I don’t know if it is relevant but my Ryzen 9 has only 12 physical cores and 24 logical cores so the detection does not work properly.

I’ve tried to compile forcing a minor number of threads using in BuildConfiguration.xml, I’ve tried first with 12 but got same errors. Then with 4 and got same errors but… less?
Finally I’ve tried with only 1, It took days to compile but NO ERRORS.

Now, finally it seems I’ve found some clues about my compilation problems, but simply I cannot compile with only one thead because it takes like forever…
Actually I don’t have any idea on how to solve. Any suggestion?

By the way I’ve checked cpuinfo and the number of physiscal/logical cores is correct at a system level.

29

Maybe I finally managed to solve this.
After yet another failed attempt with the last release I made a desperate attempt and started my Debian Linux with “nosmt” option in the bootloader, disabling hyper-threading.
This way, the build process shows only the actual 12 physical cores and not the logical ones. The build with all 12 cores in parallel went well almost to the end, then the build process was stuck with 1 core at 100% forcing me to restart the PC.
After the restart I relaunched make cmd and the compilation was successful without any other hitches.
U5 finally starts without problems!
It took me about a year to figure out that it was some kind of issue about clang/threading/cores…
Finally I can go back to work decently, I hope that this can be of some help to others too.