Having a heck of a time getting Automation AKA Unit Tests working

Development Programming & Scripting C++
1

I am having a real hard time figuring out how to get unit tests working for Unreal. I have found the very brief docs on “Automation Testing”, which seems woefully out of date. I’ve got a very simple test written, just trying to see if a UWorld can be found. It compiles (after a lot of figuring things out), but my test doesn’t show up in the Editor.

Here is my code:




#include "Economics.h"
#include "AutomationTest.h"
#include "EconomicsAgent.h"
#include "AutomationCommon.h"

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FAgentGetMarketTest, "Economics.GetMarketReference", EAutomationTestFlags::ATF_Game)

bool FAgentGetMarketTest::RunTest(const FString& Parameters)
{
	check(GEngine->GetWorldContexts().Num() == 1);
	check(GEngine->GetWorldContexts()[0].WorldType == EWorldType::Game);

	UWorld* world = GEngine->GetWorldContexts()[0].World();

	return true;
}
2

I’m having the same issue, I cannot find my test in the automation window. I tried flags ATF_Game and ATF_SmokeTest.




#include "StoryController.h"
#include "AutomationTest.h"

IMPLEMENT_SIMPLE_AUTOMATION_TEST(FStoryControllerTest, "StoryController.CreateNewGame", EAutomationTestFlags::ATF_SmokeTest)
bool FStoryControllerTest::RunTest(const FString& Parameters)
{
	return true;
}
3

Ok, I got it working for me.

ATF_Game is for tests that run during gameplay.

I don’t know why SmokeTest didn’t show up but ATK_Editor worked fine.

Tests will not hot reload so make sure to close and reopen the editor

4

But you can only access the Automation window from the editor, so how are you ever supposed to run an ATF_Game test if they don’t show up? I was assuming they would show up and in order to run them it would start the game.

In any case thanks for your comments, and I will take another stab at things this weekend.

5

Ok thanks Handkor I got it to show up and work with using ATF_Editor so that is definitely progress. However I am unsure now of how to test in a running game though since ATF_Game won’t even show up.

6

Hmm interesting. Ok if you launch your game by using the Project Launcher to build and run, then it shows up as a new Session you can connect to in the Session Frontend, and when you do that then the ATF_Game tests will show up. Apparently running PIE does not work, neither does running standalone from the play buttonm, which is a real bummer. It only seems to work if you do a complete build and launch from the Project Launcher. Having to do a complete build every time is going to slow things down :frowning:

7

Hey John, what was the latest on this? Looking into the automation system myself and was wondering if you still needed to do a full build to run ATF_Game tests. Why did you want to run with that in particular, rather than just tagging it ATF_Editor? Just making sure I’m on the same page.

Thanks!

EDIT: As a follow-up, a comment on the bottom answer of this Answerhub post says:

Another useful resource here, talking about running all automation tests for a project via cmdline, etc. Read the later answers for more details.

8

The latest for me was that I just completely gave up. Unit testing in Unreal is like pulling teeth and is such a huge pain in the rear end. The exact opposite of what it should be.

9

I was considering the following; a little C# desktop application that could pull in a list of tests for a given project, and then present them as options (or even just the overarching categories), allowing you to tick them and hit ‘test’. It then ‘builds’ the project in the selected configuration (rather than a full rebuild/cook like the Frontend, i.e. only the changes to the tests would need to be compiled) via UAT/UBT cmdline, and then runs the selected tests. Not sure how you’d get the results back - would be nice to be able to interpret them graphically, but I really don’t want to have to deal with that.

This would at least make running tests less painful, and the app wouldn’t be so crazy. There is, of course, still no mocking of classes (I’ll be making a lot of my class’ ‘brains’ as non-UObjects for this reason). Anyways, thoughts?

EDIT: Alternative; have someone make unit tests hot-reload.