Horde agent running as windows service does not invalidate horde server token

Development Pipeline & Plugins
, ,
1

After configuring and running the agent, upon viewing the logs you will see:

`

[INF] Agent capabilities queried in 2767 ms

[INF] Connecting to rpc server “https://horde.metal.gadstudios.io/”

[DBG] Starting gRPC call. Method type: ‘“Unary”’, URI: ‘“Horde”’.

[DBG] Sending message.

[VRB] Serialized ‘HordeCommon.Rpc.CreateSessionRequest’ to 916 byte message.

[VRB] Message sent.

[VRB] Response headers received.

[INF] Call failed with gRPC error status. Status code: ‘“Unauthenticated”’, Message: ‘Bad gRPC response. HTTP status code: 401’.

[DBG] Finished gRPC call.

[INF] Session failure #1. Waiting “00:00:05” and restarting. (Status(StatusCode=“Unauthenticated”, Detail=“Bad gRPC response. HTTP status code: 401”))`

Nothing I have tried will make the windows service work, including modifying any and all of the `appsettings.json` or `agent.json` files to manually include the registration token.

However, when I stop the windows service and run the agent directly from the CLI

`PS C:\Program Files\Epic Games\Horde\Agent> dotnet .\HordeAgent.dll setserver -Url=“https://horde.metal.gadstudios.io”

PS C:\Program Files\Epic Games\Horde\Agent> dotnet .\HordeAgent.dll service run

[dbg] Hosting starting

[inf] Agent capabilities queried in 2784 ms

[inf] Connecting to rpc server https://horde.metal.gadstudios.io/

[dbg] Starting gRPC call. Method type: ‘Unary’, URI: ‘Horde’.

[dbg] Sending message.

[vrb] Serialized ‘HordeCommon.Rpc.CreateSessionRequest’ to 910 byte message.

[vrb] Message sent.

[vrb] Response headers received.

[inf] Call failed with gRPC error status. Status code: ‘PermissionDenied’, Message: ‘Enrollment key does not match for DESKTOP-BT1R500’.

[dbg] Finished gRPC call.

[err] Unable to create session. Invalidating agent registration for server https://horde.metal.gadstudios.io/.

Grpc.Core.RpcException: Status(StatusCode=“PermissionDenied”, Detail=“Enrollment key does not match for DESKTOP-BT1R500”)

at Horde.Agent.Services.Session.CreateAsync(CapabilitiesService capabilitiesService, GrpcService grpcService, StatusService statusService, IOptions`1 settings, ILogger logger, CancellationToken cancellationToken) in D:\build\FN-DMH-Licensee\Sync\Engine\Source\Programs\Horde\Horde.Agent\Services\SessionFactory.cs:line 225

[14:17:26 inf] Session failure #1. Waiting 00:00:05 and restarting. (Status(StatusCode=“PermissionDenied”, Detail=“Enrollment key does not match for DESKTOP-BT1R500”))`

After this, the agent correctly invalidates the file at

`C:\Users\Will\AppData\Local\Epic Games\Horde\Agent` and proceeds as it should with registration.

Notably, if then stop this process after the agent is enrolled and restart the windows service it still fails on the original `401` seen in the first error block.

I have a few thoughts:

  • The docker image that contains the bundled tools should probably be updated. The current one is nearly a year old. (Additional question of whether there is a justification for not releasing a more recent fully bundled docker image??)
  • I had a horde agent of a newer version previously installed from a locally hosted horde server before we migrated to our new permanent one (running on an older ‘bundled’ version). I uninstalled and manually removed all traces of it before reinstalling with the one provided directly from the new horde server. I could imagine this causing a bizarre issue.
  • The windows service process is not hitting https://github.com/EpicGames/UnrealEngine/blob/2d53fcab0066b1f16dd956b227720841cad0f6f7/Engine/Source/Programs/Horde/HordeAgent/Services/SessionFactory.cs#L196 for whatever reason. I *should* see a notice of invalidation in the logs, but I do not.
2

Steps to Reproduce
Install horde-server from `https://github.com/orgs/EpicGames/packages/container/horde\-server/196738048?tag\=latest\-bundled\` and deploy.

Install tool `Horde Agent (Windows Installer)` and configure

See undesirable behavior

3

Hey Will,

Just a quick note that this docker image is significantly out of date (more than a year old - and since then we’ve had massive changes to auth/storage/etc).

I’d suggest giving https://github.com/orgs/EpicGames/packages/container/horde\-server/297562079?tag\=5\.5 a go, and for a full bundled have a look at how to produce your own docker image [Content removed] This EPS also tracks the public issue - and we are preparing the 5.6 latest bundled installer for release, as well as the 5.5 updated bundled image.

Julian

4

Hey Will,

Thanks for circling back with your findings - yes this is an issue we have caught internally and made a constituent change. Apologies on that - I should have recollected that and saved ya some time on that!

Kind regards,

Julian

5

Hi Julian,

For posterity, I was able to build the bundled horde server docker image for UE5.5.4 after the following:

  • Modifying this line in the dockerfile

dotnet tool install --global JetBrains.dotCover.GlobalTool to dotnet tool install --global JetBrains.dotCover.CommandLineToolsIt appears GlobalTool is deprecated and no longer listed on nuget. This was causing an error

  • Allocate 16GB of memory to docker desktop. Dotnet analyze will fail with OOM with the default 2GB allocation.
  • Disabling WSL2 engine in favor of HyperV in docker settings. This might be an isolated issue to my machine.
  • Ensured `Engine\Source\Programs\Horde\Scripts\test.sh` is normalized with LF line endings.

Thanks,

Will