Hello,
I would like to ask how Epic solves the problem of multi region setup when UBA is involved.
In our company we have two subnets A and B located in two different regions
Both have developers computers.
“B” is running Horde and all the agents
“A” can only access the Horde server in “B” but not the other agents, and this is thanks to an internal company VPN
My question is how Epic would solve this internally?
Do you create a giant VPN connecting all the regions together to allow a giant UBA zone or you use Tunnel/Relay mode across the globe?
I was trying to solve this first with Relay mode but discovered that nftables is mapped using only external IP addresses, so my assumption is that Relay mode should be used only for Work From Home situations, am I correct?
Then I tried to use Tunnel as well, but without any success (see crash) below
In the server.json I have the ports open correctly
I tried to run Horde Server in Debug mode with Visual Studio and setting some breakpoints and it seems the issue is that it tries to connect to port -1 without using the internal mapping.
So my final over-arching question to this problem is: is Epic using Tunnel at all to solve the multiregion issue? Using some IT-level global VPN instead? Is the UBA Zone setting involved in this in some way?
Thanks
ubt> Horde cluster resolved as 'default' ubt> Unable to get worker: System.Exception: Failed deserializing handshake response. Content: ubt> ubt> at EpicGames.Horde.Compute.Clients.TunnelHandshakeResponse.Deserialize(String text) in Engine\Source\Programs\Shared\EpicGames.Horde\Compute\Clients\ServerComputeClient.cs:line 85
Steps to Reproduce
`//server.json
“ComputeTunnelPort”: “6000”,
“ComputeTunnelAddress”: “{{ horde.address }}:6000”,
// BuildConfiguration.xml
Relay`
Hey Jaroslaw,
To answer your questions, as best as I can. As I’m most knowledge on the logistical setup of our usage and topology.
> Do you create a giant VPN connecting all the regions together to allow a giant UBA zone or you use Tunnel/Relay mode across the globe?
From a simple point of view we operate with one giant VPN. But the only classifications of ‘workstations’ on VPN who use the Direct connection mode are:
- All the build farm agents, since they are essentially on the same VPN/Network. We have a split of On-Prem UBA Helpers, for Mac, and all our Windows/Linux are in the cloud.
- We have users who have access to workstations hosted in the cloud, so we treat those as in the same ‘lan’ as the UBA helpers.
> I was trying to solve this first with Relay mode but discovered that nftables is mapped using only external IP addresses, so my assumption is that Relay mode should be used only for Work From Home situations, am I correct?
We treat WFH and folks working in an office the same. We have them use Relay mode to ensure the UBA <-> UBA traffic goes through the public internet and not through our VPN connections. But yes Relay is meant to have that HordeAgent running relay to be exposed to the internet to offload VPN traffic.
We don’t make use of the Tunnel connection mode, so it’s not tested very well currently.
A follow up answer to if we have a global UBA zone. We are fortunate enough to operate across multiple AWS regions. We have setup UBA Helpers in each of those regions. Using the compute config’s networks section, we’ve setup CIDR ranges to match all of our cloud workstations, office locations and VPN locations to route to the best matching geo-located UBA Helpers. The geo-located helpers are on ‘vpn’ to route back to the Horde server for that communication requirement.
Hope that helps a bit!
-Ryan