Automation test includes feel like they’re completely broken in 4.9.
Problem 1: Including “AutomationEditorCommon.h” breaks build
Just by including this file, my build breaks on AutomationEditorCommon.h:23 because it can’t find GLevelEditorModeTools(). This function is defined in UnrealEd.h…
Shouldn’t AutomationEditorCommon.h include UnrealEd.h as a dependency?
Problem 2: Including UnrealEd.h and calling FEditorAutomationTestUtilities::LoadMap results in linker failure
The linker can’t find FEditorFileUtils::LoadMap
This function is defined in FileHelpers.h but the module is not getting built.
Is Automation Testing simply not designed for the end binary engine user? As a base, what modules do I need to include in my game to get this stuff working?
Could you provide more info about what kind of tests you’re setting up and where in your project you’re trying to include the header? AutomationEditorCommon is aimed at testing the main editor functionality, so it’s likely that you’re either trying to do something that isn’t supported by the Binary version (changing engine-side code), or AutomationEditorCommon isn’t the right tool for your needs.
Pretty simple really. Copy code for FSetResTest straight from documentation. Hit compile, compile fails because it can’t find a function defined in AutomationEditorCommon.h. I’m including AutomationEditorCommon.h directly in the test.
The kind of tests I’m trying to setup are somewhat secondary to just getting a basic test compiling. There aren’t any test samples to scour but I’ll need to load a specific map, so the sample from the docs seems like a good starting place.
Those docs are pretty out-of-date. The current version of that test does not use the FEngineAutomationTestUtilities::LoadMap function. Additionally, FEngineAutomationTestUtilities has been renamed FEditorAutomationTestUtilities. Also, most of the methods in AutomationEditorCommon have been exported to the API, so they won’t be available outside of the UnrealEd module without making engine-level changes.
The most bare minimum compiling and passing “test” you can get is an IMPLEMENT_SIMPLE_AUTOMATION_TEST macro with a matching RunTest implementation that returns true:
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FMySimpleTest, "Project.My Simple Test", ATF_Editor | ATF_Game)
bool FMySimpleTest::RunTest(const FString& Parameters)
{
return true;
//TestTrue(TEXT("True is always..."), true); // Uncomment this line to run an actual "test"
}
So, don’t trust anything you see in the official docs, check So how would I get a pointer to the world to get my custom UGameInstance? Or can I manually construct UGameInstance or… what? It’s the code in there that I’m trying to write tests for.
I gave this a try. GEngine->GetWorld() always returns a nullptr. I tried with a level open in the editor and also while PIE. This is with the test defined as ATF_Editor. The full test:
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FPersistentSaveTest, "Arconia.PersistentSave", EAutomationTestFlags::ATF_Editor)
bool FPersistentSaveTest::RunTest(const FString& Parameters)
{
if (GEngine)
{
if (GEngine->GetWorld())
{
if (GEngine->GetWorld()->GetGameInstance())
{
return true;
}
}
else
{
AddError(FString::Printf(TEXT("Getworld is null")));
}
}
else
{
AddError(FString::Printf(TEXT("GEngine is null")));
}
return false;
}
I feel like I’m missing a fundamental step here. If I have the world, I can do pretty much anything I need.
Sorry for the bad info earlier; I’m still getting familiar with the in-game side of this.
Below is a version of the test that compiles and runs for me in game, not editor. Getting it to work in-editor will take more legwork, as you’ll need to make sure you’ve got a game running with a map loaded(eg. PIE), then iterate thru the world contexts to make sure you’re checking the right one. I’ve also reorganized it to use the built-in test-class test methods and removed the unnecessary Printf’s.
To test this, run your project with -game and -messaging flags (eg UE4Editor.exe "Path\To\Your\Project.uproject" -game -messaging), then open SessionFrontend > Automation tab and select the game session.
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FPersistentSaveTest, "CodeTest.PersistentSave", EAutomationTestFlags::ATF_Game )
bool FPersistentSaveTest::RunTest(const FString& Parameters)
{
TestNotNull(TEXT("GEngine is not null"), GEngine);
if (GEngine)
{
// Verify only one world exists
TestEqual(TEXT("GetWorldContexts returns 1"), GEngine->GetWorldContexts().Num(), 1);
if (GEngine->GetWorldContexts().Num() == 1)
{
TestNotNull(TEXT("GetGameInstance is not null"), GEngine->GetWorldContexts()[0].World()->GetGameInstance());
}
}
return true; // Any Test{Result}() failures will override this value
}
Thanks Adric, you got me to the right place. I’ve been poking around in Engine.h and am using the following to test while PIE. I can now proceed with the real testing.
IMPLEMENT_SIMPLE_AUTOMATION_TEST(FPersistentSaveTest, “Arconia.PersistentSave”, EAutomationTestFlags::ATF_Editor)
UWorld* GetWorld()
{
if (GEngine)
{
if (FWorldContext* World = GEngine->GetWorldContextFromPIEInstance(0))
{
return World->World();
}
}
return nullptr;
}
UGameInstance* GetGameInstance()
{
if (UWorld* World = GetWorld())
{
return World->GetGameInstance();
}
return nullptr;
}
bool FPersistentSaveTest::RunTest(const FString& Parameters)
{
UGameInstance* GameInstance = GetGameInstance();
TestNotNull(TEXT("GameInstance is not null"), GameInstance);
if (GameInstance)
{
// test
}
return true;
}
Would you be able to file a bug against the automation technical guide? I’d like there to be some official piece of documentation that I can send SDETs to to get started testing a game in UE4, right now this answerhub post is the best source of relevant information.
So how would I get a pointer to the world to get my custom UGameInstance? Or can I manually construct UGameInstance or… what? It’s the code in there that I’m trying to write tests for.