UE5.3 Does not find automation tests through command line

MalikGames
Tue, 12 Mar 2024 17:34:17 +0000
Programming & Scripting

I may have found a bug on Unreal Engine 5.3. Didn’t find any of this in the Issue Tracker.

I’m currently running automated tests through the command line, using the command:

“%UE5.3Folder%\Engine\Binaries\Win64\UnrealEditor-cmd.exe” “%MyProjectPath%” “%MyProjectPath%*.uproject” -nosplash -Unattended -nopause -nosound -NullRHI -nocontentbrowser -ExecCmds=“Automation RunTests Project.Functional Tests.;quit” -TestExit=“Automation Test Queue Empty” -ReportOutputPath=“%ReportFolder%” -log -Log=RunTests.log

However, the command line results in "LogAutomationCommandLine: Error: No automation tests matched ‘Project.Funcional Tests.’ ". I`m currently upgrading the project from 5.1 to 5.3, so I know this works on 5.1, but not on 5.3. If I issue the following command to see the list of automation tests (-ExecCmds=“Automation List;quit”):

“%UE5.3Folder%\Engine\Binaries\Win64\UnrealEditor-cmd.exe” “%MyProjectPath%” “%MyProjectPath%*.uproject” -nosplash -Unattended -nopause -nosound -NullRHI -nocontentbrowser -ExecCmds=“Automation List;quit” -TestExit=“Automation Test Queue Empty” -ReportOutputPath=“%ReportFolder%” -log -Log=RunTests.log

My tests are not on the list. Hence, UE5.3 does not find it.

To reproduce this bug you can:

1. Create a brand new project on UE5.3

2. Create a functional test BP and add to a map

3. Save the functional test and the map. Close unreal.

4. Run the command:
“%UE5.3Folder%\Engine\Binaries\Win64\UnrealEditor-cmd.exe” “%MyProjectPath%” “%MyProjectPath%*.uproject” -nosplash -Unattended -nopause -nosound -NullRHI -nocontentbrowser -ExecCmds=“Automation List;quit” -TestExit=“Automation Test Queue Empty” -ReportOutputPath=“%ReportFolder%” -log -Log=RunTests.log

5. Note the funcional test is not on the list of tests.

Repeat the same process for UE5.1 and you will see that the test is indeed found by the command. Hence, a bug in UE5.3.

Im missing something? There seems to have no update on this on UE5.2 and UE5.3 release notes. I dug into Unreals engine code for this and it didn`t seem to have big changes to break such thing.

Thanks in advance

3 posts - 1 participant