Launching automation test outside of the editor (on a packaged build)

anonymous_user_b9143c27
Wed, 27 Oct 2021 09:16:28 +0000
C++

Hello everyone!

Me and my team are working on a newcoming game and based on good development practices as well as Jessica Baker’s article on creating tests on unreal as well as several very good presentations about CI/CD in games using unreal. We wanted to create a workflow and pipeline to test our product all of the time and prevent a big stack of bugs during the project life cycle.

Right now we have a complete pipeline to do tests in editor but we have a hard time doing tests on a packaged build (once you’ve cooked and build and compiled your project outside of the editor)

This is very important because even when you’re working on a single platform, there’s a lot of issues that could appear in build that doesn’t show in editor.

So far we’ve explored a lot:

We took a look at gauntlet but had a very hard time setting things up since documentations and examples are very rare

We tried to find a command that would launch automation tests through UAT on a build with the right parameters but couldn’t find a command that would do that (we have a test that returns true if the test is run in build)

The thing is, I am so confused about why it’s so hard to test in build, litterally there’s a bunch of tools that seems to be made for runtime tests like:

A runtime automation test plugin for project agnostic runtime tests (how does a built project works with project agnostic runtime test, I don’t know but it’s there)

Two different blueprint type, one for functional test and one specificly for editor functional test that is not packaged with the content

Doesn’t it mean there SHOULD be a way to launch and track tests from a build?
Does anyone have any insight, ideas or clue on how to set that up properly and simply. Right now we’re trying to:

Read the source code to understand if there’s any way to launch and track tests from the build with unreal base source code

Create our own test launche that logs the results and send back a text file or an html report to track in build tests

If anyone could help it would be very much appreciated.

3 posts - 3 participants