Using RunUnreal in a Horde BuildGraph cannot find automation project, even when passed in with the -scriptdir command

Development Pipeline & Plugins
, ,
1

I have an automation project included in my project’s Plugins folder that adds multiple UnrealTestRoles for an Editor, a Server, and multiple clients (depending on arguments provided). I am able to run tests that use this automation project when calling the RunUAT.bat command with a RunUnreal call, for example:

D:\[ProjectRootDirectory]\Engine\Build\BatchFiles\RunUAT.bat RunUnreal -scriptdir=D:\[ProjectRootDirectory]\[Game]\[Game]\Plugins\MyCustomAutomation\Source\MyCustomAutomation.Automation -project=D:\[ProjectRootDirectory]\[Game]\[Game]\[Game].uproject -test=MyCustomAutomation.SimpleCustomAutomationTest -runtest=“Project.MyTestToRun” -DedicatedClientNum=2 -platform=Win64 -configuration=Development -build=D:\[PathToPackagedServerAndClientBuilds] -maxDuration=3000 -LogIdleTimeout=3000

UAT is able to find the automation project that way and the test runs as expected.

When I try to call this in Build Graph using “<Command Name=“RunUnreal” Arguments=…” I get a namespace issue:

“Unable to find type MyCustomAutomation.SimpleCustomAutomationTest in assemblies. Namespaces=Gauntlet.UnrealTest, UnrealGame, UnrealEditor.”

The arguments are the same as above, but it seems the Horde Agent’s UAT isn’t building the custom automation project. I have tried including a node to build the engine and UAT:

<Node Name=“Compile $(EditorTarget) Win64” Requires=“Generating Project Files” Produces=“#EditorBinaries”>

<Compile Target=“$(EditorTarget)” Platform=“Win64” Configuration=“Development” Project=“$(ProjectFile)” Tag=“#EditorBinaries”/>

<Spawn Exe=“powershell.exe” Arguments=“$(RootDir)\Engine\Build\BatchFiles\BuildUAT.bat quiet FORCE&quot;”/>

</Node>

in the build graph before trying to call RunUnreal but that hasn’t helped. Is there something I’m doing wrong? Should I just be calling RunUAT.bat similar to how I am calling BuildUAT.bat above to make sure it’s using the project version of the Automation Tool or am I misunderstanding what’s going wrong here?

[Attachment Removed]

2

Steps to Reproduce

  • Set up an Automation project that is set up to run Gauntlet tests
    • This project has a test that adds UnrealTestRoles for an engine, server, and multiple clients running on the same machine
    • The automation project is located in the Plugins folder of the game project
  • Set up Horde to build the project
    • Horde is able to build the engine and game of the project
  • In a build graph for a Horde agent to run, include a call “<Command Name=“RunUnreal” Arguments=…” that references a test in the namespace of the custom Automation project
    • In the Arguments, the path to the automation project is included in -scriptdir=&quot;$(ProjectPath)\Plugins\MyCustomAutomation\Source\MyCustomAutomation.Automation&quot;
  • Run the build graph
  • Observe the error: “Unable to find type MyCustomAutomation.SimpleCustomAutomationTest in assemblies. Namespaces=Gauntlet.UnrealTest, UnrealGame, UnrealEditor.”
    [Attachment Removed]
3

I got an answer to this myself, actually.

The solution is in the project.stream.json file, when defining the job as a template, in the arguments (where the build graph is referenced by -Script=) include the -scriptdir= argument there and the call to RunUAT that Horde does will recognize the automation project there. I think my confusion came from which instance of the Automation Tool was running the build graph and where that got built.

[Attachment Removed]

4

Hi, good work on solving this.

Matthew

[Attachment Removed]