UBA cook and shader compilation on HordeAgent

Development Pipeline & Plugins
, ,
1

Hello

We are integrating UBA in our infrastructure. We have Horde server running on AWS Linux (docker), a windows machine and a Linux Horde agent all in the same subnet. We trigger the build from windows machine that has the project and builds successfully locally.

We have configured the following settings:

Engine/Saved/UnrealBuildTool/BuildConfiguration.xml:

`<?xml version="1.0" encoding="utf-8" ?>

true http://{{ SERVER_HOST_NAME }}:13340

Win-UE5

true true

``Config\DefaultEngine.ini`

`[UbaController]
Enabled=true
bForceRemote=true
Horde=(Pool=Linux-UE5,MaxCores=500)

[Horde]
ServerUrl=“http://{{ SERVER_HOST_NAME }}:13340/”`

We can see that compilation is done remotely and the agent gets CPU high load, but in case of cook process, the main load is carried by windows machine though we can see the following messages:

... LogDesktopPlatform: Display: Launching OidcToken... [C:/UE5/Engine/Binaries/DotNET/OidcToken/win-x64/OidcToken.exe --HordeUrl="http://<ip>:13340/" --OutFile="../../../../j_work_zspace/Intermediate/oidcToken.json54B6B43B4341E9BDCC35A18F7.tmp" --Unattended=true] ... LogShaderCompilers: Display: Using UBA Controller for Shader Compilation. ...

We are using the following command and arguments:

"$UE5_PATH\Engine\Build\BatchFiles\RunUAT.bat`" -ScriptsForProject=`“${PROJECT_PATH}`” BuildCookRun -nocompileeditor -installed -unattended -nop4 -project=`“${PROJECT_PATH}`” -cook -allmaps -stage -pak -archive -archivedirectory=`“${ARCHIVEDIRECTORY_CLIENT}`” -package -unrealexe=`“${UE5_PATH}\Engine\Binaries\Win64\UnrealEditor-Cmd.exe`” -ddc=InstalledDerivedDataBackendGraph -prereqs -nodebuginfo -targetplatform=Win64 -build -target=`“${PROJECT_NAME}`” -clientconfig=${CLIENTCONFIG} -utf8output -createreleaseversion=‘1.0’"

Our goal is to use bigger Linux agent and smaller windows which triggers the build for cost optimization.

Is there something we can do to use Linux agent more heavily, because in our case it does very little during cooking and mainly is idle?

Or maybe there is another approach not to keep the agent mainly idle during the whole process?

Regards

2

Steps to Reproduce

3

Hey Narek,

The only real mechanism you have at your disposal is the pooling mechanisms (it looks like you have a Linux-UE5 & Win-UE5). How I’d separate this is have all your helpers annotated within a UBA specific pool, and have your primary build machines (the thing kicking off the build) specified as the main pool for your template. These machines will then initiate the build on your build farm (and then instigate a UBA distributed build through the other agents via the corresponding pool). Let me know if I’m not understanding your question properly.

Kind regards,

Julian

4

Hey Narek,

Ah understood. Cooking cannot be offloaded to remote via UBA. Currently compile & shader compilation are the only items distributable via UBA.

Julian

5

Hey Julian

I think there’s a misunderstanding.

My question is specifically about the distribution of workload between the primary build machine and the helper agents. I’m trying to understand why the Linux agent remains mostly idle during the cooking process while the Windows machine carries most of the load. My goal is to shift more work to the Linux agent for better utilization and cost optimization.

Could you clarify whether cooking can be offloaded similarly to compilation, and if so, how to improve the distribution?

Best Regards