Having problem running pixel streaming in docker: failed to create a CUDA context

Lordontcare · July 12, 2022, 2:42pm

I have packed the project, tested ok in Windows., cross-compile linux,
however, when deploying in a ubuntu docker,
I encounter following error (when running with xxx_project.sh -RenderOffscreen -PixelStreamingIP=localhost -PixelStreamingPort=8888) :

[2022.07.12-14.25.08:410][  0]LogInit: Display: Game Engine Initialized.
[2022.07.12-14.25.08:410][  0]LogCUDA: Display: Initialising CUDA API...
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: CUDA API initialised successfully.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found 7 CUDA capable devices.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 0 called NVIDIA A100-PCIE-40GB.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 1 called NVIDIA Quadro RTX 8000.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 2 called NVIDIA Quadro RTX 8000.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 3 called NVIDIA Quadro RTX 8000.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 4 called NVIDIA Quadro RTX 8000.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 5 called NVIDIA Quadro RTX 8000.
[2022.07.12-14.25.08:509][  0]LogCUDA: Display: Found device 6 called NVIDIA Quadro RTX 8000.
Failed to find symbol file, expected location:
"/home/hmp/fuqingxu/UHMP/Build/LinuxNoEditor/UHMP/Binaries/Linux/UHMP.sym"
Fatal error: [File:F:/UnrealSourceCode/UnrealEngine-4.27.2-release/Engine/Source/Runtime/CUDA/Source/Private/CudaModule.cpp] [Line: 202] 
CUDA module failed to create a CUDA context on the RHI selected device with UUID 2010431984.
[2022.07.12-14.25.08:512][  0]LogCore: Error: appError called: Fatal error: [File:F:/UnrealSourceCode/UnrealEngine-4.27.2-release/Engine/Source/Runtime/CUDA/Source/Private/CudaModule.cpp] [Line: 202] 
CUDA module failed to create a CUDA context on the RHI selected device with UUID 2010431984.
0x0000000008bc370b UHMP!UnknownFunction(0x89c370a)
0x0000000008bc5dbf UHMP!UnknownFunction(0x89c5dbe)
0x0000000003362b9c UHMP!UnknownFunction(0x3162b9b)
0x0000000003354063 UHMP!UnknownFunction(0x3154062)
0x00000000033629c0 UHMP!UnknownFunction(0x31629bf)
0x0000000008b56d69 UHMP!UnknownFunction(0x8956d68)
0x00007f17d57d0083 libc.so.6!__libc_start_main(+0xf2)
0x0000000003353649 UHMP!UnknownFunction(0x3153648)




Signal 11 caught.
Malloc Size=65538 LargeMemoryPoolOffset=65554 
CommonUnixCrashHandler: Signal=11
[2022.07.12-14.25.08:524][  0]LogCore: === Critical error: ===
Unhandled Exception: SIGSEGV: invalid attempt to write memory at address 0x0000000000000003

[2022.07.12-14.25.08:524][  0]LogCore: Fatal error: [File:F:/UnrealSourceCode/UnrealEngine-4.27.2-release/Engine/Source/Runtime/CUDA/Source/Private/CudaModule.cpp] [Line: 202] 
CUDA module failed to create a CUDA context on the RHI selected device with UUID 2010431984.

please help, I don’t know what to do with this crash …

detail___
the docker contrainer is launched by

docker run -itd   --name  uebuild-$USER \
--runtime=nvidia \
 -e NVIDIA_DRIVER_CAPABILITIES=all \
--net host \
--memory 400G \
--memory-reservation 400G \
--volume /var/run/docker.sock:/var/run/docker.sock \
--gpus all \
--shm-size=16G \
my_nv_docker_image

And the game is launched by :

my_project.sh  -RenderOffscreen -nosound  -PixelStreamingIP=localhost -PixelStreamingPort=8888

Whole startup log:
UHMP.log (62.5 KB)

end of detail____

pengchaoJay · June 27, 2023, 8:09am

i meet the same question，i don’t know what to do

ismael0016 · July 14, 2023, 1:12am

Hello guys, I’m facing the same issue

root-project-1     | [2023.07.13-18.45.36:399][  0]LogInit: Display: Game Engine Initialized.
root-project-1     | [2023.07.13-18.45.36:399][  0]LogCUDA: Display: Initialising CUDA API...
root-project-1     | [2023.07.13-18.45.37:022][  0]LogCUDA: Display: CUDA API initialised successfully.
root-project-1     | [2023.07.13-18.45.37:022][  0]LogCUDA: Display: Found 1 CUDA capable devices.
root-project-1     | [2023.07.13-18.45.37:022][  0]LogCUDA: Display: Found device 0 called NVIDIA GeForce RTX 3070.
root-project-1     | [2023.07.13-18.45.37:027][  0]LogCore: Error: appError called: Fatal error: [File:/home/ue4/UnrealEngine/Engine/Source/Runtime/CUDA/Source/Private/CudaModule.cpp] [Line: 202]
root-project-1     | CUDA module failed to create a CUDA context on the RHI selected device with UUID 166714944.
root-project-1     | 0x0000000008b5751b FirstPersonProject!UnknownFunction(0x895751a)
root-project-1     | 0x0000000008b59bcf FirstPersonProject!UnknownFunction(0x8959bce)
root-project-1     | 0x000000000335763c FirstPersonProject!UnknownFunction(0x315763b)
root-project-1     | 0x0000000003348b03 FirstPersonProject!UnknownFunction(0x3148b02)
root-project-1     | 0x0000000003357460 FirstPersonProject!UnknownFunction(0x315745f)
root-project-1     | 0x0000000008aeab79 FirstPersonProject!UnknownFunction(0x88eab78)
root-project-1     | 0x00007fccaf201d90 libc.so.6!UnknownFunction(0x29d8f)
root-project-1     | 0x00007fccaf201e40 libc.so.6!__libc_start_main(+0x7f)
root-project-1     | 0x00000000033480e9 FirstPersonProject!UnknownFunction(0x31480e8)
root-project-1     |
root-project-1     |
root-project-1     |
root-project-1     |
root-project-1     | CommonUnixCrashHandler: Signal=11
root-project-1     | [2023.07.13-18.45.37:039][  0]LogCore: === Critical error: ===
root-project-1     | Unhandled Exception: SIGSEGV: invalid attempt to write memory at address 0x0000000000000003
root-project-1     |
root-project-1     | [2023.07.13-18.45.37:039][  0]LogCore: Fatal error: [File:/home/ue4/UnrealEngine/Engine/Source/Runtime/CUDA/Source/Private/CudaModule.cpp] [Line: 202]
root-project-1     | CUDA module failed to create a CUDA context on the RHI selected device with UUID 166714944.
root-project-1     | 0x0000000008b5751b FirstPersonProject!UnknownFunction(0x895751a)
root-project-1     | 0x0000000008b59bcf FirstPersonProject!UnknownFunction(0x8959bce)
root-project-1     | 0x000000000335763c FirstPersonProject!UnknownFunction(0x315763b)
root-project-1     | 0x0000000003348b03 FirstPersonProject!UnknownFunction(0x3148b02)
root-project-1     | 0x0000000003357460 FirstPersonProject!UnknownFunction(0x315745f)
root-project-1     | 0x0000000008aeab79 FirstPersonProject!UnknownFunction(0x88eab78)
root-project-1     | 0x00007fccaf201d90 libc.so.6!UnknownFunction(0x29d8f)
root-project-1     | 0x00007fccaf201e40 libc.so.6!__libc_start_main(+0x7f)
root-project-1     | 0x00000000033480e9 FirstPersonProject!UnknownFunction(0x31480e8)

… did you found something ?