When trying to run the engine under Linux with the Vulkan backend and the -RenderOffscreen command line flag, the engine ends up passing a nullptr as the SDL_Window* into SDL_Vulkan_CreateSurface. This happens in VulkanLinuxPlatform.cpp:
void FVulkanLinuxPlatform::CreateSurface(void* WindowHandle, VkInstance Instance, VkSurfaceKHR* OutSurface)
{
EnsureSDLIsInited();
if (SDL_Vulkan_CreateSurface((SDL_Window*)WindowHandle, Instance, OutSurface) == SDL_FALSE)
{
UE_LOG(LogInit, Error, TEXT("Error initializing SDL Vulkan Surface: %s"), SDL_GetError());
check(0);
}
}
Here’s the command that I’m using to launch the engine:
Engine/Binaries/Linux/UE4Editor /home/efagerho/git/UE4Upstream/Samples/StarterContent/StarterContent.uproject -game -MovieSceneCaptureType="/Script/MovieSceneCapture.AutomatedLevelSequenceCapture" -LevelSequence="/Game/Sequence" -MovieFolder="/home/efagerho/ue4/output/" -NoLoadingScreen -MovieFormat=JPG -MovieQuality=100 -ResX=1920 -ResY=1080 -vulkan -RenderOffscreen
I’ve added a trivial Sequence into the StarterContent project, so that I could record some data. It’s pretty clear that this hits a code path that hasn’t been tested at all. Basically, if the RenderOffscreen flag is being provided, then in SlateApplication.cpp we see that MakeWindow contains the following:
if (bRenderOffScreen)
{
TSharedRef< FGenericWindow > NewWindow = MakeShareable(new FGenericWindow());
InSlateWindow->SetNativeWindow(NewWindow);
return NewWindow;
}
However, the implementation of GetOSWindowHandle() in FGenericWindow is simply simply “return nullptr;”. This is what is given to the CreateSurface() method above.
I ended up implementing fully headless rendering myself over the last couple of days. In other words, I rewrote some of the Vulkan Linux platform code and made use of the following Vulkan layer:
Basically, this allows me to create a fake VkSurfaceKHR object that I can pass to the engine, so that headless rendering can be implemented without having to touch larger parts of the Vulkan renderer.