Unreal Tests on Github Actions with Test Reporter

Development Platform & Builds
1

I figured out how to use Github actions to build and test Unreal game projects with C++ Automation tests.

The keys to it are:

  • Use an official Unreal Engine docker image
  • Have the correct authentication set up
  • Figure out good build and test scripts
  • Use a test reporter that handles the Unreal Engine test output JSON

For that last one I had to write my own because it seems it didn’t exist yet. See below.

Previously folks said this could not be done, as in a 2025 post:

You’re not gonna be able to do it entirely within github actions. You’ll need a self-hosted runner. This is because storage is limited on GH actions runners. Your scripts will fail during pre-requisite setup because they’re guaranteed to exceed GH storage quotas.

Using the Docker image gets around this because Unreal is already built.

container:
      image: ghcr.io/epicgames/unreal-engine:dev-slim-5.6.1
      credentials:
        username: ${{ secrets.GHCR_IO_USERNAME }}
        password: ${{ secrets.REPO_ACCESS_TOKEN }}

This is a snippet from the test script that runs in the container:

# Outputs results in JSON format - can we get dart-json? https://github.com/dart-lang/test/blob/master/pkgs/test/doc/json_reporter.schema.json
/home/ue4/UnrealEngine/Engine/Binaries/Linux/UnrealEditor "${PROJECT_DIR}/${PROJECT_NAME}.uproject" \
   -execcmds="Automation RunTests ${TESTS};Quit" \
   -stdout -unattended -NOSPLASH -AllowStdOutLogVerbosity -NullRHI \
   -ReportExportPath="${PROJECT_DIR}/Reports" \
   -AbsLog="${PROJECT_DIR}/Logs" \
   -TestExit="Automation Test Queue Empty"

This is the reporter which examines the output of the tests and passes or fails the build:

      - name: Results
        if: always()
        uses: sarah-j-smith/test-reporter-unreal@v3.1.3
        with:
          name: Unreal Engine Automation Tests
          path: Reports/*.json
          reporter: unreal-json

I’ve contacted the maintainer of the test reporter project with a PR to merge my fork, but no reply as per yet. However its simple enough to just use my fork as it is (as in the YAML above).

I kind of went down a bit of a rabbit hole with the CI stuff and now want to get back to game dev, but I’d be interested in hearing from other folks who might try this to see how it works for them.