Running Tests Jenkins

[meiwen]

Hi,

I am currently setting up my build system using automated tests. We are running a jenkins server that checksout the latest dev branch, runs static coda analysis, packages the game and deploys it to steam every night. Now I want to automated tests.
Any tipps on a better setup highly appreciated
I wrote some basic tests, like checking if the correct map, gamemode, etc is set. If I run the tests via command line. The tests run fine. But if I use the exact same command with Jenkins I get this error:

UE4Editor: [2017.01.19-14.41.27:143][  0]LogDerivedDataCache:Display: Pak cache opened for reading ../../../Engine/DerivedDataCache/Compressed.ddp.
UE4Editor: [2017.01.19-14.41.31:609][  0]LogD3D11RHI:Error: D3DRHI->GetFactory()->CreateSwapChain(DXGIDevice,&SwapChainDesc,SwapChain.GetInitReference()) failed 
UE4Editor:  at D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\Windows\WindowsD3D11Viewport.cpp:57 
[2017.01.19-14.41.31:609][  0]LogD3D11RHI:Error: D3DRHI->GetFactory()->CreateSwapChain(DXGIDevice,&SwapChainDesc,SwapChain.GetInitReference()) failed 
 at D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\Windows\WindowsD3D11Viewport.cpp:57 
 with error 887A0022
[2017.01.19-14.41.31:622][  0]LogWindows: Windows GetLastError: Der Vorgang wurde erfolgreich beendet. (0)
UE4Editor:  with error 887A0022
[2017.01.19-14.41.32:160][  0]LogWindows:Error: begin: stack for UAT
UE4Editor: [2017.01.19-14.41.32:160][  0]LogWindows:Error: begin: stack for UAT
[2017.01.19-14.41.32:163][  0]LogWindows:Error: === Critical error: ===
UE4Editor: [2017.01.19-14.41.32:163][  0]LogWindows:Error: === Critical error: ===
[2017.01.19-14.41.32:173][  0]LogWindows:Error: 
UE4Editor: [2017.01.19-14.41.32:173][  0]LogWindows:Error: 
[2017.01.19-14.41.32:183][  0]LogWindows:Error: Fatal error: [File:D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\D3D11Util.cpp] [Line: 227] 
UE4Editor: [2017.01.19-14.41.32:183][  0]LogWindows:Error: Fatal error: [File:D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\D3D11Util.cpp] [Line: 227] 
[2017.01.19-14.41.32:193][  0]LogWindows:Error: D3DRHI->GetFactory()->CreateSwapChain(DXGIDevice,&SwapChainDesc,SwapChain.GetInitReference()) failed 
UE4Editor: [2017.01.19-14.41.32:193][  0]LogWindows:Error: D3DRHI->GetFactory()->CreateSwapChain(DXGIDevice,&SwapChainDesc,SwapChain.GetInitReference()) failed 
[2017.01.19-14.41.32:203][  0]LogWindows:Error:  at D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\Windows\WindowsD3D11Viewport.cpp:57 
UE4Editor: [2017.01.19-14.41.32:203][  0]LogWindows:Error:  at D:\Build\++UE4+Release-4.14+Compile\Sync\Engine\Source\Runtime\Windows\D3D11RHI\Private\Windows\WindowsD3D11Viewport.cpp:57 
[2017.01.19-14.41.32:213][  0]LogWindows:Error:  with error 887A0022
UE4Editor: [2017.01.19-14.41.32:213][  0]LogWindows:Error:  with error 887A0022
[2017.01.19-14.41.32:223][  0]LogWindows:Error: 
UE4Editor: [2017.01.19-14.41.32:223][  0]LogWindows:Error: 
[2017.01.19-14.41.32:233][  0]LogWindows:Error: 
UE4Editor: [2017.01.19-14.41.32:233][  0]LogWindows:Error: 
[2017.01.19-14.41.32:243][  0]LogWindows:Error: KERNELBASE.dll
UE4Editor: [2017.01.19-14.41.32:243][  0]LogWindows:Error: KERNELBASE.dll
[2017.01.19-14.41.32:253][  0]LogWindows:Error: UE4Editor-Core.dll!FOutputDeviceWindowsError::Serialize() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\windows\windowsplatformoutputdevices.cpp:103]
UE4Editor: [2017.01.19-14.41.32:253][  0]LogWindows:Error: UE4Editor-Core.dll!FOutputDeviceWindowsError::Serialize() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\windows\windowsplatformoutputdevices.cpp:103]
[2017.01.19-14.41.32:263][  0]LogWindows:Error: UE4Editor-Core.dll!FOutputDevice::Logf__VA() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\misc\outputdevice.cpp:68]
UE4Editor: [2017.01.19-14.41.32:263][  0]LogWindows:Error: UE4Editor-Core.dll!FOutputDevice::Logf__VA() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\misc\outputdevice.cpp:68]
[2017.01.19-14.41.32:273][  0]LogWindows:Error: UE4Editor-Core.dll!FDebug::AssertFailed() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\misc\assertionmacros.cpp:332]
UE4Editor: [2017.01.19-14.41.32:273][  0]LogWindows:Error: UE4Editor-Core.dll!FDebug::AssertFailed() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\core\private\misc\assertionmacros.cpp:332]
[2017.01.19-14.41.32:283][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!VerifyD3D11Result() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\d3d11util.cpp:227]
UE4Editor: [2017.01.19-14.41.32:283][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!VerifyD3D11Result() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\d3d11util.cpp:227]
[2017.01.19-14.41.32:293][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!FD3D11Viewport::FD3D11Viewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\windows\windowsd3d11viewport.cpp:60]
UE4Editor: [2017.01.19-14.41.32:293][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!FD3D11Viewport::FD3D11Viewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\windows\windowsd3d11viewport.cpp:60]
[2017.01.19-14.41.32:303][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!FD3D11DynamicRHI::RHICreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\d3d11viewport.cpp:473]
UE4Editor: [2017.01.19-14.41.32:303][  0]LogWindows:Error: UE4Editor-D3D11RHI.dll!FD3D11DynamicRHI::RHICreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\windows\d3d11rhi\private\d3d11viewport.cpp:473]
[2017.01.19-14.41.32:313][  0]LogWindows:Error: UE4Editor-SlateRHIRenderer.dll!RHICreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\rhi\public\dynamicrhi.h:1328]
UE4Editor: [2017.01.19-14.41.32:313][  0]LogWindows:Error: UE4Editor-SlateRHIRenderer.dll!RHICreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\rhi\public\dynamicrhi.h:1328]
[2017.01.19-14.41.32:323][  0]LogWindows:Error: UE4Editor-SlateRHIRenderer.dll!FSlateRHIRenderer::CreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\slaterhirenderer\private\slaterhirenderer.cpp:277]
UE4Editor: [2017.01.19-14.41.32:323][  0]LogWindows:Error: UE4Editor-SlateRHIRenderer.dll!FSlateRHIRenderer::CreateViewport() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\slaterhirenderer\private\slaterhirenderer.cpp:277]
[2017.01.19-14.41.32:333][  0]LogWindows:Error: UE4Editor-SlateCore.dll!SWindow::ShowWindow() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\slatecore\private\widgets\swindow.cpp:1209]
UE4Editor: [2017.01.19-14.41.32:333][  0]LogWindows:Error: UE4Editor-SlateCore.dll!SWindow::ShowWindow() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\slatecore\private\widgets\swindow.cpp:1209]
[2017.01.19-14.41.32:344][  0]LogWindows:Error: UE4Editor-Engine.dll!UGameEngine::CreateGameWindow() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\engine\private\gameengine.cpp:420]
UE4Editor: [2017.01.19-14.41.32:344][  0]LogWindows:Error: UE4Editor-Engine.dll!UGameEngine::CreateGameWindow() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\engine\private\gameengine.cpp:420]
[2017.01.19-14.41.32:354][  0]LogWindows:Error: UE4Editor-MoviePlayer.dll!FDefaultGameMoviePlayer::Initialize() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\movieplayer\private\defaultgamemovieplayer.cpp:147]
UE4Editor: [2017.01.19-14.41.32:354][  0]LogWindows:Error: UE4Editor-MoviePlayer.dll!FDefaultGameMoviePlayer::Initialize() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\movieplayer\private\defaultgamemovieplayer.cpp:147]
[2017.01.19-14.41.32:364][  0]LogWindows:Error: UE4Editor.exe!FEngineLoop::PreInit() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\launchengineloop.cpp:1629]
UE4Editor: [2017.01.19-14.41.32:364][  0]LogWindows:Error: UE4Editor.exe!FEngineLoop::PreInit() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\launchengineloop.cpp:1629]
[2017.01.19-14.41.32:374][  0]LogWindows:Error: UE4Editor.exe!GuardedMain() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\launch.cpp:113]
UE4Editor: [2017.01.19-14.41.32:374][  0]LogWindows:Error: UE4Editor.exe!GuardedMain() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\launch.cpp:113]
[2017.01.19-14.41.32:384][  0]LogWindows:Error: UE4Editor.exe!GuardedMainWrapper() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:126]
UE4Editor: [2017.01.19-14.41.32:384][  0]LogWindows:Error: UE4Editor.exe!GuardedMainWrapper() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:126]
[2017.01.19-14.41.32:394][  0]LogWindows:Error: UE4Editor.exe!WinMain() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:202]
UE4Editor: [2017.01.19-14.41.32:394][  0]LogWindows:Error: UE4Editor.exe!WinMain() [d:\build\++ue4+release-4.14+compile\sync\engine\source\runtime\launch\private\windows\launchwindows.cpp:202]
[2017.01.19-14.41.32:404][  0]LogWindows:Error: UE4Editor.exe!__scrt_common_main_seh() [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:264]
UE4Editor: [2017.01.19-14.41.32:404][  0]LogWindows:Error: UE4Editor.exe!__scrt_common_main_seh() [f:\dd\vctools\crt\vcstartup\src\startup\exe_common.inl:264]
[2017.01.19-14.41.32:414][  0]LogWindows:Error: KERNEL32.DLL
UE4Editor: [2017.01.19-14.41.32:414][  0]LogWindows:Error: KERNEL32.DLL
[2017.01.19-14.41.32:424][  0]LogWindows:Error: ntdll.dll
UE4Editor: [2017.01.19-14.41.32:424][  0]LogWindows:Error: ntdll.dll
[2017.01.19-14.41.32:434][  0]LogWindows:Error: ntdll.dll
UE4Editor: [2017.01.19-14.41.32:434][  0]LogWindows:Error: ntdll.dll
[2017.01.19-14.41.32:444][  0]LogWindows:Error: 
UE4Editor: [2017.01.19-14.41.32:444][  0]LogWindows:Error: 
[2017.01.19-14.41.32:454][  0]LogWindows:Error: end: stack for UAT
[2017.01.19-14.41.32:490][  0]LogExit: Executing StaticShutdownAfterError
[2017.01.19-14.41.32:495][  0]LogWindows: FPlatformMisc::RequestExit(1)
[2017.01.19-14.41.32:504][  0]Log file closed, 01/19/17 15:41:32
UE4Editor: [2017.01.19-14.41.32:454][  0]LogWindows:Error: end: stack for UAT

I guess this might have something to do with the granted rights to the jenkins user? Some ideas how to avoid this?

Cheers Pascal

[WoodsShin]

Jenkins is running as a service with LOCALSERVICE account. In my case, node.js is running to execute an application on the same machine.

[Myralogue]

Hey, I'm currently trying to get this to work but can't seem to figure out a way. Could you maybe elaborate how you got this working using node.js?
I've tried installing the NodeJS plugin into Jenkins, but I get the same result as when running it as a bat or Windows Powershell script i.e. The window doesn't show up becuase it's ran in non interactive mode, and the test isn't ran (I'm using 'child_process' btw).

Thanks Martijn

[arnobn]

Hello , i'm currently trying to solve this problem someone have found a solution ?

[Manoel.Neto]

You cannot run tests in non-interactive mode because UE4 will try (and fail) to create a viewport.

[arnobn]

I find a solution if you add "-nullrhi" to the command line unreal will be execute in background

[Muzaheed]

did any of u get success to do android packaging using jenikins?


my jenkin build always failing into "Execution failed for task ':app:mergeReleaseResources'. phase.

[xxEoD2242]

I find a solution if you add "-nullrhi" to the command line unreal will be execute in background

This did the trick for me when trying to run Unreal to do a specific job on AWS. Thanks!

[Sameer.Mirza]

Hi meiwen, did you find a solution for this? I'm also trying to run some tests on Jenkins where I require RHI so can't really use -nullrhi either.