The check to avoid assigning distributed builds to the requester’s machine does not work properly in HordeServer.Compute.ComputeService.

Development Pipeline & Plugins
, ,
1

(This is a translation of a [Japanese [Content removed] by Kokawa Takashi.)

When requesting a distributed build using UBA via HordeServer, I noticed that the same machine as the requester is selected.

Looking into the code, I confirmed that around line 418 of the following file, a condition has been added to avoid assigning work to the same machine as the requester:

Engine\Source\Programs\Horde\Plugins\Compute\HordeServer.Compute\Compute\ComputeService.cs

When investigating the cause in the debug execution environment described in the reproduction steps, which is described in the following post, I noticed that requesterIp contained an IPv6-mapped IPv4 address such as “::ffff:127.0.0.1”. The comparison condition was ComputeIp != ‘::ffff:127.0.0.1’, which caused the comparison with the IPv4 ComputeIp to fail.

I also noticed the following in the related areas:

■ HordeServer side

In Engine\Source\Programs\Horde\HordeServer\Commands\ServerCommand.cs, the KestrelServer is started using the ListenAnyIP method. In this configuration, listening on IPv6 is the default behavior. In the HordeServer execution log where I noticed the issue, I also confirmed from the following log entry that it was listening on IPv6:

Now listening on: [http://[::]:13340](http://[::]:13340)

Then, in Engine\Source\Programs\Horde\Plugins\Compute\HordeServer.Compute\Compute\ComputeController.cs, the requester IP address obtained via

RequesterIp = HttpContext.Connection.RemoteIpAddress seems to be returned as an IPv6-mapped IPv4 address in the environment, and it is then passed to the comparison condition that causes the issue.

■ HordeAgent side

In Engine\Source\Programs\Horde\HordeAgent\Services\CapabilitiesService.cs, the ComputeIp value is set, and appears to be handled as an IPv4 address.

ComputeIp can also be set in the HordeAgent configuration file (Agent.json). So, I tried specifying IPv6 addresses and IPv6-mapped IPv4 addresses there as well, but it did not work. The issue were not resolved.

== Questions ==

(1) Is there a configuration-based workaround for this issue (environment configuration or adjustments to the HordeServer / HordeAgent settings)?

(2) What network configuration is expected for Horde?

- Is HordeServer expected to listen on IPv4 or IPv6?

- What value is expected for HordeAgent ComputeIp?

(3) In what situations should ComputeIp be adjusted manually?

== Request ==

I have confirmed that the issue can be resolved by modifying the code so that the handling of IPv6/v4 matches our environment.

So, as a fundamental solution, I would like to request that Epic Games consider implementing support in favor of such network environment patterns.

== Other related information ==

The following discussion appears to be relevant:

[Content removed]

> A) When a compute resource (agent) is requested, it will not assign work to an agent matching requesting user’s IP

I guess the issue occurs because the above mechanism does not work.

Thanks.

[Attachment Removed]

2

[Reproduction steps]

I have confirmed that the issue can be reproduced in the following two environments:

- A HordeServer/HordeAgent environment installed using a Windows MSI installer built from the Horde Server source code.

- A HordeServer/HordeAgent environment where HordeServer (+ HordeDashboard) is started in debug mode by opening Engine\Source\Programs\Horde\Horde.sln

Setup

- Connect the local machine to the running HordeServer as a HordeAgent.

- Specify the HordeServer in BuildConfiguration.xml.

- Set the Horde Agent operating mode to Dedicated.

Build

Open the Visual Studio solution of any UE project (for example, the engine itself or Lyra) and start the build.

Behavior during the build

The build log shows the connection to the target HordeServer, and the worker connection log indicates that the local machine is selected and operates as a RemoteExecutor.

An excerpt from the build log is shown below:

Horde URL: http://{OUR_HORDE_SERVER}:13340/, Pool: (none), Cluster: (none), Condition: (none), Connection: (none), Encryption: (none), MaxCores: 576, MaxWorkers: 64, MaxIdle: 15s
Horde cluster resolved as 'default'
[Worker0] Connected to {OTHER_HOST1} ({OTHER_HOST1_IPv4_ADDRESS}) under lease 69b35025822aa0f059c3fec7 (agent version: 5.7.3-0)
[Worker1] Connected to {OTHER_HOST2} ({OTHER_HOST2_IPv4_ADDRESS}) under lease 69b35025822aa0f059c3fece (agent version: 5.7.3-0)
[Worker2] Connected to {OTHER_HOST3} ({OTHER_HOST3_IPv4_ADDRESS}) under lease 69b35025822aa0f059c3fed2 (agent version: 5.7.3-0)
[Worker3] Connected to {MY_HOST} ({MY_HOST_IPv4_ADDRESS}) under lease 69b35025822aa0f059c3fed8 (agent version: 5.7.3-0)
 
** For UnrealEditor-Win64-Development **
[1/41930] Compile [x64] BulkDataTests.cpp [RemoteExecutor: {MY_HOST}]
...
[25/41930] Compile [x64] EditorPathHelper.cpp [RemoteExecutor: {OTHER_HOST1}]
...
[44/41930] Compile [x64] ClassDefaultObjectTest.cpp [RemoteExecutor: {OTHER_HOST3}]

The host names and IP addresses have been replaced with placeholders. The portion labeled {MY_HOST} represents the machine that requested the build being selected as a worker.

(As additional note about the versions used: the engine version requesting the distributed build (UBT) was tested with 5.7.3, 5.6.1 and 5.7.1, and the same issue was seen.)

[Attachment Removed]

3

Hi, thank you for the bug report. I will raise an issue with the horde team to get this fixed.

Matthew

[Attachment Removed]

4

[mention removed]​

(This is a translation of a Japanese post by Kokawa Takashi.)

I understand and thank you for the bug report and support.

I’d appreciate it if you could let me know when the bug report is public.

[Attachment Removed]

5

Hi, to answer your other questions

1. Is there a configuration-based workaround?

  • On the agent side, explicitly set ComputeIp in Agent.json to a plain IPv4 address: { “Horde”: { “ComputeIp”: “192.168.1.100” } } This bypasses auto-detection entirely.
  • There is no server-side configuration knob, Kestrel uses ListenAnyIP() which is hardcoded to bind dual-stack.

2. What network configuration is expected?

  • HordeServer listens on both IPv4 and IPv6 by default via ListenAnyIP()
  • HordeAgent ComputeIp is expected to be a plain IPv4 address or left unset for auto-detection. The auto-detection method GetLocalIpAddressAsync creates an AddressFamily.InterNetwork (IPv4-only) socket, which always returns a plain IPv4 address. It cannot produce IPv6-mapped addresses. The format mismatch originates entirely on the server side: Kestrel’s dual-stack listener returns HttpContext.Connection.RemoteIpAddress as an IPv6-mapped IPv4 address (e.g. ::ffff:192.168.1.1) for IPv4 connections, which then fails string comparison against the agent’s plain IPv4 ComputeIp.

3. When should ComputeIp be set manually?

  • The agent has multiple network interfaces and the auto-detected one (based on routing to the server) isn’t the one other agents should use for compute connections
  • The auto-detected IP is incorrect for your environment, verify by checking the agent’s ComputeIp property in the Horde dashboard and confirming it matches the expected reachable address
  • Most deployments should leave it unset

Matthew

[Attachment Removed]

6

[mention removed]​

Is there any update with the Issue you raised?

Thanks

[Attachment Removed]

7

Hi, a fix has been made and will be released in 5.8. The change can be seen at https://github.com/EpicGames/UnrealEngine/commit/94158508bbba8bf795f374556abf119565855e8e

Matthew

[Attachment Removed]