Building lightmaps on a headless box

I’m trying to set up an automated CI build process for my project, running on a headless Linux build machine, and the only thing not working is building lightmaps. The command I’m using is:

/opt/UnrealEngine/Engine/Binaries/Linux/UE4Editor /path/to/MyProject.uproject -run=resavepackages -buildlighting -mapsonly -projectonly -allowcommandletrendering -messaging -nullrhi

I applied a small hack to my UE4 code (built on the build box) to answer “yes” to the “do you want to rebuild missing/incompatible modules?” dialog that pops up - since on this box it can’t show the dialog and assumes No.

It reaches the point where it would start to run Lightmass - the output of the command is sparse, but from delving into the code and planting debug logging, I’ve determined this to be FLightmassMaterialRenderer::CreateRenderTarget - and then crashes. The reason for the crash is because a new render target resource is never actually created due to the app not being able to render anything… because of the -nullrhi argument.

Without -nullrhi, however, it crashes even earlier, but at least this leaves me with a decent amount of info:

[2018.09.02-21.04.42:853][  0]LogInit: Initializing SDL.
[2018.09.02-21.04.42:912][  0]LogInit: Initialized SDL 2.0.5 revision: 11184 (hg-11184:04063928c4a8 with EpicExtensions) (compiled against 2.0.5)
[2018.09.02-21.04.42:913][  0]LogInit: Using SDL_WINDOW_OPENGL
[2018.09.02-21.04.42:913][  0]LogInit: Using SDL video driver 'offscreen'
[2018.09.02-21.04.42:913][  0]LogInit: Display metrics:
[2018.09.02-21.04.42:913][  0]LogInit:   PrimaryDisplayWidth: 1024
[2018.09.02-21.04.42:913][  0]LogInit:   PrimaryDisplayHeight: 768
[2018.09.02-21.04.42:913][  0]LogInit:   PrimaryDisplayWorkAreaRect:
[2018.09.02-21.04.42:913][  0]LogInit:     Left=0, Top=0, Right=1024, Bottom=768
[2018.09.02-21.04.42:913][  0]LogInit:   VirtualDisplayRect:
[2018.09.02-21.04.42:913][  0]LogInit:     Left=0, Top=0, Right=1024, Bottom=768
[2018.09.02-21.04.42:913][  0]LogInit:   TitleSafePaddingSize: X=0.000 Y=0.000 Z=0.000 W=0.000
[2018.09.02-21.04.42:913][  0]LogInit:   ActionSafePaddingSize: X=0.000 Y=0.000 Z=0.000 W=0.000
[2018.09.02-21.04.42:913][  0]LogInit:   Number of monitors: 1
[2018.09.02-21.04.42:913][  0]LogInit:     Monitor 0
[2018.09.02-21.04.42:913][  0]LogInit:       Name: 0
[2018.09.02-21.04.42:913][  0]LogInit:       ID: display0
[2018.09.02-21.04.42:913][  0]LogInit:       NativeWidth: 1024
[2018.09.02-21.04.42:913][  0]LogInit:       NativeHeight: 768
[2018.09.02-21.04.42:913][  0]LogInit:       bIsPrimary: true
Signal 11 caught.
Malloc Size=131076 LargeMemoryPoolOffset=131092 
CommonUnixCrashHandler: Signal=11
Malloc Size=65535 LargeMemoryPoolOffset=196655 
Malloc Size=502624 LargeMemoryPoolOffset=699296 
Malloc Size=68112 LargeMemoryPoolOffset=767424 
[2018.09.02-21.04.42:946][  0]LogCore: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to read memory at address 0x0000000000000010

[2018.09.02-21.04.42:946][  0]LogCore: Fatal error!

0x00007f65bd13320a libUE4Editor-ApplicationCore.so!OFFSCREEN_CreateWindow()
0x00007f65bd129cb7 libUE4Editor-ApplicationCore.so!SDL_CreateWindow_REAL()
0x00007f6521ed6c7a libUE4Editor-OpenGLDrv.so!Linux_PlatformCreateDummyGLWindow(FPlatformOpenGLContext*) [/opt/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/Linux/OpenGLLinux.cpp:119]
0x00007f6521ed98dd libUE4Editor-OpenGLDrv.so!PlatformInitOpenGL() [/opt/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/Linux/OpenGLLinux.cpp:856]
0x00007f6521efe315 libUE4Editor-OpenGLDrv.so!FOpenGLDynamicRHI::FOpenGLDynamicRHI() [/opt/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp:1154]
0x00007f6521efe0a2 libUE4Editor-OpenGLDrv.so!FOpenGLDynamicRHIModule::CreateRHI(ERHIFeatureLevel::Type) [/opt/UnrealEngine/Engine/Source/Runtime/OpenGLDrv/Private/OpenGLDevice.cpp:1137]
0x00007f65bd81e01c libUE4Editor-RHI.so!PlatformCreateDynamicRHI() [/opt/UnrealEngine/Engine/Source/Runtime/RHI/Private/Linux/LinuxDynamicRHI.cpp:68]
0x00007f65bd81d587 libUE4Editor-RHI.so!RHIInit(bool) [/opt/UnrealEngine/Engine/Source/Runtime/RHI/Private/DynamicRHI.cpp:180]
0x00000000002330b6 UE4Editor!FEngineLoop::PreInit(wchar_t const*) [/opt/UnrealEngine/Engine/Source/Runtime/Launch/Private/LaunchEngineLoop.cpp:1806]
0x0000000000243bf8 UE4Editor!GuardedMain(wchar_t const*) [/opt/UnrealEngine/Engine/Source/Runtime/Launch/Private/Launch.cpp:42]
0x00007f65b8717fdb libUE4Editor-UnixCommonStartup.so!CommonUnixMain(int, char**, int (*)(wchar_t const*)) [/opt/UnrealEngine/Engine/Source/Runtime/Unix/UnixCommonStartup/Private/UnixCommonStartup.cpp:245]
0x00007f65b8161b87 libc.so.6!__libc_start_main(+0xe6)
0x0000000000230029 UE4Editor!_start()

My digging revealed that’s because - being a headless server - it can’t actually render anything anyway… which is what led to me learning to use nullrhi in the first place!

Before using this command, I tried using the RebuildLightmaps automation script, but for some reason it requires P4. This person ran into the same issue a couple years ago, and managed to fix it by hacking out the P4 dependency in the script - but since that time P4 has been baked much harder in there and it’s not as straightforward to remove.

I’m at a loss at this point… does anyone know how to successfully build lighting headlessly? Any hints would be greatly appreciated!

EDIT: I took another look at the RebuildLightMaps automation command, and discovered at least 4 glaring problems with it.

  1. It has a bunch of P4 stuff baked in. I was able to remove it, though perhaps less cleanly than ideal.
  2. It’s hard coded to build its dependencies for Win64. I changed it to Linux for my purposes.
  3. It’s hard coded to always delete and rebuild its dependencies for some reason. I flipped that off for my purposes.
  4. It doesn’t accept any extra arguments to send to the commandlet. I added -messaging for my purposes.

Those changes made it run properly on my machine, but as it turns out (as I sort of expected) it’s just a wrapper around the commandlet and fails in the same spot. Oh well.