“Uba has a feature called zones. What these do is that when multiple helpers from the same zone connects to a host, then one helper becomes a proxy for the others for certain traffic (downloading files). When using AWS/googlecloud this happens automatically (if you look in the Uba source code and UbaAWS.h).. otherwise you can solve this by using environment variable UBA_ZONE or on command line with -zone”
I have searched everywhere and I cannot find any documentation or help on the topic.
How is this setup and enabled?
Also, is there documentation (or will there be) on UBA Cache Server?
There isn’t any documentation for this unfortunately. I’ve looked around internally, and the feature effectively works as indicated by Henrik:
Instruction works as follows:
UBA_ZONE is set automatically in the AWS context
Outside of this, you can specify the environment variable UBA_ZONE on the agent or -zone
Underlying behaviour as follows:
If host and helpers are in same zone, they can download cas files directly without a proxy
If a helper is in a different zone, then the first one connecting from that zone will also create a proxy
There is a special consideration here:
If a host has any character different than the last character, then it will see all zones that rare matching each other except the last character as the same zone
This means that if a host with us_west1a connects to helpers with zones us-west1b and us-west1c, those will all have separate proxies
But if host julianLocalZone connects to us-west1a and us-west1b, then all us-west1<X> will be seen as the same zone
This applies for your bespoke custom zones outside of AWS/GCP
julianLocalZone-easta & julianLocalZone-eastb - the same rules apply
General guidance is if you locally host your UBA helpers, to have your zones be per office.
This feature stands outside of Horde (much of UBA does, as a matter of fact - the primary Horde contribution being the Horde UBA Coordinator).
Unfortunately beyond this, we don’t have much documentation.
UbaAgent.exe has “-noproxy” which can be used to make sure this agent never becomes the proxy for others. There is no code in horde to control this however.. I can add an environment variable check for UBA_ALLOW_PROXY=0 if you want so you can add that to the machines you don’t want to be proxy. I guess for the future maybe we need to add this support in the horde UI
The zones are kind of designed to be used on a LAN when there are machines outside the LAN that wants to use the LAN machines as helpers. If there is latency or bandwidth limitations between machines in the same zone, then the zone is probably setup wrong.
So if I’m working from home and my office is somewhere else and there are twenty machines in the office that could help my machine build. In that case you want those twenty machines to be in the same zone.
In practice what will happen is that my machine will connect to each of the twenty machines (or the other way around depending on how your setup is).. the helpers will tell my machine what zone they belong in.. so now I know there are x helpers that is in a specific zone.
The assignment of proxy is basically first machine in a zone requesting a file. Since the first request can be a huge pch, we don’t want to wait and see if there is a “better” machine to be a proxy… because then it would mean that the pch needs to be transferred multiple times while the decision is made. When second machine requests a file, the server will tell the second machine which machine is the proxy, the second machine will then connect to the proxy and request the same file through there instead.
If the machine that is the proxy goes down, then the next machine requesting a file becomes the proxy.
Is this the hidden key to making UBA reasonably usable across widely separated networks?
I feel like this needs to be better documented, and maybe some sort of default thing set for those that just stand up horde and throw some agents at it.
Going to try this on our network as soon as I can get a chance to test it
Yes, especially when using precompiled headers which are often really big. If you WFH and have access to an office with lots of machines.. then you can send over your pch once and then they are shared inside the office.
I live in Vancouver and the closest AWS data center we’re using is in Oregon somewhere.. I have the core limit set to 600 cores.. so I get around 25 machines helping me.. the first machine that asks for a file gets told to become the proxy. Then when second machine asks for a file it gets the answer to use the first machine as proxy.. it then connect via LAN to the first machine and ask again for the file but this time via the first machine.
So instead of transferring a 300mb (2gb uncompressed) pch 25 times, I just transfer it once.
Not letting first connection be a proxy is a bit tricky because of the big files. As soon as one machine declines you will have two downloads of that 300mb file.. so waiting for a specific machine to become a proxy is not really a good idea since you likely will send that 300mb file multiple times.
There is code in UBA to build and use an explicit proxy (UbaStorageProxy target).. but I have not tested that in a year or so and it is probably not working… if there is a desire to have dedicated proxy instances then I can probably quite quickly fix it up.
yes. very handy I build Lyra in 3 minutes from clean from home.
There has been external teams out there comparing uba with the alternatives and one studio claims a 2.5x build time speedup with the exact same hardware when switching over… We haven’t really compared uba with the other solutions but I can tell you that the reason I started implementing uba was because the other solutions didn’t work well over latent connections.
would the zone be simply setting the environment variable UBA_ZONE=anynameZone on all agents and in BuildConfiguration UnrealBuildAccelerator set Zone=anynameZone?
If so, say we have dedicated agents and workstation agents, what do I do so that workstation agents help, but do not become proxies?
If some agents are better in being proxy than others (not sure on requirements) then I believe that a setting in Horde UI would be nice.
Setting parameters or variables on a lot of agents and restarting the HordeAgent service takes a bit of time and the Horde UI would be simple and quicker.
I imagine that would be quite handy for studios that are primarily remote, and are not using AWS by default. Especially with a little bit of documentation pointing in the direction of it
As soon as the person with the keys to Ansible gets back around, i am going to try this, I am super interested in seeing how this works compared to SNDBS and Incredibuild and icecc and others from full remote.
I build Lyra in 3 minutes from clean from home.