Connection to external devices not working in Gauntlet after updating to UE5.5

Development Pipeline & Plugins
, ,
1

After updating from UE5.4 to UE5.5, the connection to test devices stopped working when running tests via Gauntlet.

I’m using this command in a BuildGraph script:

RunUnreal -Project=E:/MyGame/MyGame.uproject -ScriptDir=E:/MyGame -Test=UE.TargetAutomation -RunTest=.Perf. -ReportExportPath=E:/MyGame/Saved/Tests -Build=E:/MyGame/build -Platform=SomeConsole -Configuration=Development -Device=SomeConsole:11.22.33.44 -Device=SomeConsole:55.66.77.88 -ResumeOnCriticalFailureThe logs show this error:

Not enough devices of type SomeConsole:SomeConsole:55.66.77.88 exist for test. (1 required, 0 available)This led me to this commit:

https://github.com/EpicGames/UnrealEngine/commit/50e57f8d612ff98f9728eac2ae49c5b22c17837e

After debugging, I applied a hotfix by reverting changes in:

Engine/Source/Programs/AutomationTool/Gauntlet/Unreal/RunUnreal.csfrom:

TestContext.Constraint = new UnrealDeviceTargetConstraint(UnrealPlatform, PerfSpec, Model, DeviceName);to:

TestContext.Constraint = new UnrealDeviceTargetConstraint(UnrealPlatform, PerfSpec, Model);

Question:

Is there a better way to fix this issue? Has this been reported as a known bug?

2

The `-Device=<deviceName>` argument is meant to force the Gauntlet Device reservation to use specific registered devices when a pool of devices is provided. It should be the name used in the registry; the port should not be included.

But I believe you are trying to specify what devices are available to the Device reservation. In that case the proper command line argument should be `-Devices=SomePlatform:ADDRESS1,SomePlatform:ADDRESS2`

3

Ok, I think I’m seeing the issue more clearly now. There is a clash of argument parsing. Some arguments are really close and were used by different part historically… I’ll have this sorted out. Thanks for highlighting the issue.