Pixel Streaming Linux Docker container crashes on startup

Robotdjman · July 1, 2024, 4:40pm

Hey,

Our team has had a working Pixel Streaming build on Windows for quite sometime now and it now we have been prepping to get this feature into our production pipeline which requires us getting it packaged into a Linux Docker container. Unfortunately we have been running into this crash on startup and I can’t seem to figure out what the issue is:

Below is a snippet including the error

2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Display: Found VULKAN_SDK=
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Display: Registering provided Vulkan validation layers
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Display: Updated VK_LAYER_PATH=../../../Engine/Binaries/ThirdParty/Vulkan/Linux
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Display: Updated LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu:/usr/lib/i386-linux-gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:../../../Engine/Binaries/ThirdParty/Vulkan/Linux
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogInit: Using SDL_WINDOW_VULKAN
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Display: Built with Vulkan header version 1.3.239
2024-06-26 18:13:09 [2024.06.27-00.13.09:637][  0]LogVulkanRHI: Starting Vulkan Profile check for VP_UE_Vulkan_SM5_RT:
2024-06-26 18:13:09 [2024.06.27-00.13.09:639][  0]LogVulkanRHI: Error: vpCreateInstance(&ProfileInstanceCreateInfo, VULKAN_CPU_ALLOCATOR, &TempInstance) failed, VkResult=-9
2024-06-26 18:13:09  at .\Runtime/VulkanRHI/Private/VulkanRHI.cpp:296 
2024-06-26 18:13:09  with error VK_ERROR_INCOMPATIBLE_DRIVER
2024-06-26 18:13:09 [2024.06.27-00.13.09:643][  0]LogCore: Error: appError called: Fatal error: [File:.\Runtime/VulkanRHI/Private/VulkanUtil.cpp] [Line: 1131] 
2024-06-26 18:13:09 vpCreateInstance(&ProfileInstanceCreateInfo, VULKAN_CPU_ALLOCATOR, &TempInstance) failed, VkResult=-9
2024-06-26 18:13:09  at .\Runtime/VulkanRHI/Private/VulkanRHI.cpp:296 
2024-06-26 18:13:09  with error VK_ERROR_INCOMPATIBLE_DRIVER
2024-06-26 18:13:09 0x000000000895e06c Landscape4v23!VulkanRHI::VerifyVulkanResult(VkResult, char const*, char const*, unsigned int) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanUtil.cpp:1130]
2024-06-26 18:13:09 0x0000000008917bda Landscape4v23!CheckVulkanProfile(ERHIFeatureLevel::Type, bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:296]
2024-06-26 18:13:09 0x000000000891893b Landscape4v23!FVulkanDynamicRHI::CreateInstance() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:657]
2024-06-26 18:13:09 0x000000000891872e Landscape4v23!FVulkanDynamicRHI::FVulkanDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:508]
2024-06-26 18:13:09 0x0000000008917d3a Landscape4v23!FVulkanDynamicRHIModule::CreateRHI(ERHIFeatureLevel::Type) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:379]
2024-06-26 18:13:09 0x0000000008a276ec Landscape4v23!PlatformCreateDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/Linux/LinuxDynamicRHI.cpp:135]
2024-06-26 18:13:09 0x0000000008a2506e Landscape4v23!RHIInit(bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/DynamicRHI.cpp:322]
2024-06-26 18:13:09 0x000000000b6549e0 Landscape4v23!FEngineLoop::PreInitPreStartupScreen(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/LaunchEngineLoop.cpp:3201]
2024-06-26 18:13:09 0x000000000b64d6e0 Landscape4v23!GuardedMain(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/Launch.cpp:140]
2024-06-26 18:13:09 0x000000000b5ab64e Landscape4v23!CommonUnixMain(int, char**, int (*)(char16_t const*), void (*)()) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Unix/UnixCommonStartup/Private/UnixCommonStartup.cpp:269]
2024-06-26 18:13:09 0x00007fc37df29083 libc.so.6!__libc_start_main(+0xf2)
2024-06-26 18:13:09 0x000000000492b029 Landscape4v23!_start()
2024-06-26 18:13:09 
2024-06-26 18:13:09 
2024-06-26 18:13:09 
2024-06-26 18:13:09 
2024-06-26 18:13:09 CommonUnixCrashHandler: Signal=11
2024-06-26 18:13:09 [2024.06.27-00.13.09:652][  0]LogCore: === Critical error: ===
2024-06-26 18:13:09 Unhandled Exception: SIGSEGV: invalid attempt to write memory at address 0x0000000000000003
2024-06-26 18:13:09 
2024-06-26 18:13:09 [2024.06.27-00.13.09:652][  0]LogCore: Fatal error: [File:.\Runtime/VulkanRHI/Private/VulkanUtil.cpp] [Line: 1131] 
2024-06-26 18:13:09 vpCreateInstance(&ProfileInstanceCreateInfo, VULKAN_CPU_ALLOCATOR, &TempInstance) failed, VkResult=-9
2024-06-26 18:13:09  at .\Runtime/VulkanRHI/Private/VulkanRHI.cpp:296 
2024-06-26 18:13:09  with error VK_ERROR_INCOMPATIBLE_DRIVER
2024-06-26 18:13:09 0x000000000895e06c Landscape4v23!VulkanRHI::VerifyVulkanResult(VkResult, char const*, char const*, unsigned int) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanUtil.cpp:1130]
2024-06-26 18:13:09 0x0000000008917bda Landscape4v23!CheckVulkanProfile(ERHIFeatureLevel::Type, bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:296]
2024-06-26 18:13:09 0x000000000891893b Landscape4v23!FVulkanDynamicRHI::CreateInstance() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:657]
2024-06-26 18:13:09 0x000000000891872e Landscape4v23!FVulkanDynamicRHI::FVulkanDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:508]
2024-06-26 18:13:09 0x0000000008917d3a Landscape4v23!FVulkanDynamicRHIModule::CreateRHI(ERHIFeatureLevel::Type) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:379]
2024-06-26 18:13:09 0x0000000008a276ec Landscape4v23!PlatformCreateDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/Linux/LinuxDynamicRHI.cpp:135]
2024-06-26 18:13:09 0x0000000008a2506e Landscape4v23!RHIInit(bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/DynamicRHI.cpp:322]
2024-06-26 18:13:09 0x000000000b6549e0 Landscape4v23!FEngineLoop::PreInitPreStartupScreen(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/LaunchEngineLoop.cpp:3201]
2024-06-26 18:13:09 0x000000000b64d6e0 Landscape4v23!GuardedMain(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/Launch.cpp:140]
2024-06-26 18:13:09 0x000000000b5ab64e Landscape4v23!CommonUnixMain(int, char**, int (*)(char16_t const*), void (*)()) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Unix/UnixCommonStartup/Private/UnixCommonStartup.cpp:269]
2024-06-26 18:13:09 0x00007fc37df29083 libc.so.6!__libc_start_main(+0xf2)
2024-06-26 18:13:09 0x000000000492b029 Landscape4v23!_start()
2024-06-26 18:13:09 
2024-06-26 18:13:09 
2024-06-26 18:13:09 
2024-06-26 18:13:09 0x000000000895e06c Landscape4v23!VulkanRHI::VerifyVulkanResult(VkResult, char const*, char const*, unsigned int) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanUtil.cpp:1130]
2024-06-26 18:13:09 0x0000000008917bda Landscape4v23!CheckVulkanProfile(ERHIFeatureLevel::Type, bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:296]
2024-06-26 18:13:09 0x000000000891893b Landscape4v23!FVulkanDynamicRHI::CreateInstance() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:657]
2024-06-26 18:13:09 0x000000000891872e Landscape4v23!FVulkanDynamicRHI::FVulkanDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:508]
2024-06-26 18:13:09 0x0000000008917d3a Landscape4v23!FVulkanDynamicRHIModule::CreateRHI(ERHIFeatureLevel::Type) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/VulkanRHI/Private/VulkanRHI.cpp:379]
2024-06-26 18:13:09 0x0000000008a276ec Landscape4v23!PlatformCreateDynamicRHI() [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/Linux/LinuxDynamicRHI.cpp:135]
2024-06-26 18:13:09 0x0000000008a2506e Landscape4v23!RHIInit(bool) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/RHI/Private/DynamicRHI.cpp:322]
2024-06-26 18:13:09 0x000000000b6549e0 Landscape4v23!FEngineLoop::PreInitPreStartupScreen(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/LaunchEngineLoop.cpp:3201]
2024-06-26 18:13:09 0x000000000b64d6e0 Landscape4v23!GuardedMain(char16_t const*) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Launch/Private/Launch.cpp:140]
2024-06-26 18:13:09 0x000000000b5ab64e Landscape4v23!CommonUnixMain(int, char**, int (*)(char16_t const*), void (*)()) [C:/Users/metca/Git/UnrealEngine/Engine/Source/./Runtime/Unix/UnixCommonStartup/Private/UnixCommonStartup.cpp:269]
2024-06-26 18:13:09 0x00007fc37df29083 libc.so.6!__libc_start_main(+0xf2)
2024-06-26 18:13:09 0x000000000492b029 Landscape4v23!_start()
2024-06-26 18:13:09 
2024-06-26 18:13:09 [2024.06.27-00.13.09:652][  0]LogExit: Executing StaticShutdownAfterError
2024-06-26 18:13:09 Engine crash handling finished; re-raising signal 11 for the default handler. Good bye.

So far this is just using Docker desktop with a Windows 11 host. I have an NVIDIA 3080 Ti and the GPU looks to be successfully passed through into the container as well as the Linux driver according to me running nvidia-smi inside the container:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.52.01              Driver Version: 555.99         CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3080 ...    On  |   00000000:01:00.0  On |                  N/A |
| N/A   51C    P0             35W /  162W |     425MiB /  16384MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I also ran several test GPU docker containers and they seemed to work fine (docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark).

Any help is appreciated!

Robotdjman · July 2, 2024, 9:29pm

It looks like currently Docker Desktop does not support many of the APIs required for Vulkan to run on Docker Desktop for windows. Windows only provides limited functionality (enough to get a good chunk of CUDA to work) of the GPU to the WSL2 VM causing this to crash. Same docker compose and docker image starts without crashing on a dedicated Linux VM with an NVIDIA GPU, NVIDIA Drivers, and NVIDIA Container Toolkit installed.

system · August 1, 2024, 9:30pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.