In Engine/Source/Runtime/Engine/Private/World.cpp : 2784 - bool UWorld::SetGameMode(const FURL& InURL)
if( GIsAutomationTesting )
{
// fall back to raw GameMode if Automation Testing, as the shared engine maps were not designed to use what could be any developer default GameMode
GameClass = AGameMode::StaticClass();
}
This looks like a hack, and precludes doing any sort of testing of gameplay features (i.e. regression testing pathfinding or object interactions).
Eliminating that code path and allowing the game mode to be loaded normally during Automation Testing didn’t cause any issues with any of the tests bundled with the engine.