Clarification on Horde Device Manager (HDM) Support for Third-party Mobile Devices

Development Pipeline & Plugins
, ,
1

We’re currently exploring integrating our Gauntlet-based client tests with mobile devices from third-party cloud providers for mobile test execution. However, we’ve identified some limitations with Horde Device Manager (HDM) related to device connectivity and networking.

This is what I learned from Gaunlet’s (android) device reservation code:

  1. Gauntlet uses Horde’s device reservation APIs to reserve devices and retrieve their hostname/IP address. It also handles automatic lease renewal.
  2. It checks whether a device with the same hostname/IP is already connected locally.
  3. If not, it attempts to connect using ADB:
    1. `adb connect <hostname>:5555` (if the hostname doesn’t include a port)
    2. `adb connect <hostname>` (if the hostname includes a port)
  4. If the connection is successful, Gauntlet proceeds with the test using the now-local device.

This suggests the horde agent can only work with Android devices that 1) has adb tcpip enabled and 2) directly accessible in the same network.

Could you please confirm:

  • Does HDM only officially support on-premises mobile devices that are directly accessible through our corporate network?
  • If third-party devices are not accessible via the corporate network (thus limiting direct ADB-over-network access), what would be the recommended approach for integrating these external devices with Gauntlet tests?
  • Does Epic have any plans to extend HDM functionality to natively support third-party device providers in the future?

Specifically, would a custom BuildGraph node or specialized Gauntlet commands, which handle reserving devices via HDM and then manage connectivity through third-party tools/APIs, be the appropriate solution?

Your guidance on the best practices for integrating such third-party device providers would be greatly appreciated.

2

Steps to Reproduce

3

Hi there Jason,

I’ve followed up with the team internally on your question.

  1. HDM isn’t strictly opinionated about this - it’s effectively a db to track devices and their status (and when poked via Gauntlet, provide a satiating device). That being said, we do not make use of third-party devices at Unreal, so I can’t speak to any pre-existing configuration that would work.
  2. Gauntlet is the code of import here (as you’ve indicated).
  3. Whilst this feature is not currently being actively developed, it has come up in discussion before, and there are some existing investigation efforts around this.

Regarding a potential workaround, I do think it best to solve this in the gauntlet command side. HDM can be naively interpreted to be a key value pair system of devices, and you could introduce some “translation” logic once you receive the “go-ahead” from HDM. This is how I’d attempt to tackle this problem at the moment in order to “fully” obtain the device on the gauntlet side. This way, it keeps the HDM solution as the ‘semaphore’ holder, and gauntlet as the “actual connection to device”.

I hope that helps a bit; if you have any suggestions or find a pattern that works, I’d highly recommend bringing it back to this EPS thread, or perhaps even a PR.

Kind regards,

Julian

4

Hey Jason,

Would you mind creating a separate EPS for this, and link it back here? It’s a great question, and I want to make sure it gets indexed properly in search.

Julian

5

Thank you very much for your confirmation.

This aligns with our understandings. I’ll update this thread to let you know how it goes.

6

Hi Julian

Following up the previous one, I have a couple of questions regarding device testing and reservation workflows:

1) Multi-device mobile testing

How is testing across multiple mobile devices typically managed? Specifically:

- How many devices can be connected to a single agent at once?

- Can tests be run sequentially or in parallel across multiple connected devices?

One scenario we are considering is:

- Running performance tests across a large fleet of Android devices (e.g., 30 devices).

- Running similar performance tests across iOS devices (e.g., 31 devices).

We’d like to understand the best practices and limitations around test execution models and device management in this context.

2) Device lease renewal and failover handling

Once devices are reserved, we’d like to maintain those leases reliably. In particular:

- Is there a built-in mechanism to renew device reservations _in the background_ while tests are still running?

- This would help us handle situations such as agent machine failure, where we would prefer to release or reclaim devices proactively.

I also noticed BuildGraph nodes can use annotations like `Annotations=“DeviceReserve=Begin”`, but couldn’t find documentation on how these annotations behave. Specifically:

- Does this annotation ensure that device reservations are maintained for the duration of the node execution?

- Is it possible to explicitly manage reservation lifetimes using annotations or another mechanism?

Thanks in advance for your help and guidance.

Jason

7

Here it is: [Content removed]

Thanks!