How to get Horde Agent to use HTTP/2.0 to connect to gRCP server

When I try to set up my Horde Agents, they appear to be trying to connect to the server using HTTP/1.1 which is not supported by the server (see the error message below).

I am hosting the server on-prem in our local k8s cluster and am installing the agents on our build machines. The documentation suggests it should be as simple as installing the agent software and setting the server URL, but on each machine, the agent appears to be trying to use the wrong protocol.

I have adjusted our ingress to specifically route traffic from the agents to port 5002 and still have the same issue. I’ve also manually configured the server profile in the Agent’s appsettings.json file to no avail.

Has anyone experienced this or have any advice on how to get my agents up and running? Thanks in advance!

Error:

Session failure #608. Waiting "00:01:00" and restarting. (Status(StatusCode="Internal", Detail="Request protocol 'HTTP/1.1' is not supported."))
Grpc.Core.RpcException: Status(StatusCode="Internal", Detail="Request protocol 'HTTP/1.1' is not supported.")
   at Grpc.Net.Client.Internal.Retry.RetryCallBase`2.GetResponseCoreAsync()
   at HordeAgent.Services.Session.RegisterAgentAsync(GrpcService grpcService, AgentSettings agentSettings, RpcAgentCapabilities capabilities, ILogger logger, CancellationToken cancellationToken) in D:\build\RHLicensee\Sync\Engine\Source\Programs\Horde\HordeAgent\Services\SessionFactory.cs:line 269
   at HordeAgent.Services.Session.CreateAsync(CapabilitiesService capabilitiesService, GrpcService grpcService, StatusService statusService, IOptions`1 settings, HordeClientFactory hordeClientFactory, ILogger logger, CancellationToken cancellationToken) in D:\build\RHLicensee\Sync\Engine\Source\Programs\Horde\HordeAgent\Services\SessionFactory.cs:line 160
   at HordeAgent.Services.SessionFactory.CreateAsync(CancellationToken cancellationToken) in D:\build\RHLicensee\Sync\Engine\Source\Programs\Horde\HordeAgent\Services\SessionFactory.cs:line 363
   at HordeAgent.Services.WorkerService.ExecuteInnerAsync(CancellationToken stoppingToken) in D:\build\RHLicensee\Sync\Engine\Source\Programs\Horde\HordeAgent\Services\WorkerService.cs:line 130

HTTP/2.0 mandates TLS, and your ingress is likely not configured to to handle this use case. IIRC, you must either:

  • Configure Horde server with certificate information so it can terminate TLS on its own and serve all traffic on the same port (443)
  • Have your ingress terminate TLS, and forward gRPC traffic to the server as HTTP/2 Cleartext (h2c in most reverse proxies), or
  • Leave everything unencrypted, and use a separate port for gRPC traffic from the agents. The ingress must not attempt to negotiate HTTP/2 on the server’s behalf, and also accept h2c.