Choosing an automated test solution from the provided Unreal frameworks

Development Pipeline & Plugins
, ,
1

Hello!

I have a few questions about automated tests in Unreal.

For our project we are starting to write automated tests and we want to use some framework to make our lives easier. The context for what we are trying to achieve:

  • Tests are hierarchically grouped in test suites, we can define dependencies between the suites.
  • We have a strong inclination for using some sort of Command system. By using commands we hope it will speed up writing the tests and it should be easier to make sure they are valid.
  • We would like the command system / commands to be available in blueprints as we will have scripters and designer writing the tests as well
  • We can pass parameters/metadata to the tests. The metadata is hierarchical, e.g. you can have a group of test share data, and then a subset of tests add on top of it
  • We will be running the tests on Horde.

I’ve noticed there are three different ways to define tests:

  1. The Automation Testing Framework (ATF) simple/complex automation tests,
  2. ATF Spec tests,
  3. CQ Tests

Questions

  1. Are they all just alternatives to each other or do the different test definitions have specific use-cases?
  2. Is either of them more important, i.e. it will be maintained and improved more over the others?
  3. What would be your recommendation for the requirements we have?

[Attachment Removed]

2

All the automated tests written in UE C++ use the ATF ( as you call it ) as base.

The different types are different flavors or set of harnesses to help writing tests.

CQTests is the latest of the iteration and the one that is still actively being develop. But we are maintaining all of them for legacy reason.

CQTests have some helper functions to simplify working with BP assets. That said, if you plan to use BP mainly. The BP functional tests are probably better suited for this. https://dev.epicgames.com/documentation/en\-us/unreal\-engine/functional\-testing\-in\-unreal\-engine

[Attachment Removed]

3

Thank you for the information! We are going to be using a lot of code tests, but also use functional tests, so it will be a mix of both. We will investigate the CQTests as the blueprint helpers could really be useful.

[Attachment Removed]