Intermittent UBA Horde failures during shader compilation in cook (works on retry)

Development Pipeline & Plugins
,
1

Hi,

We’re currently integrating UBA for both C++ and shader compilation in Unreal Engine 5.6, and we’re hitting intermittent failures during shader compilation in cook.

Context

  • Unreal Engine version: 5.6

  • Horde / UBA agents: 5.7

  • UBA enabled for:

    • C++ compilation (working reliably)

    • Shader compilation (enabled, but unstable)

  • Cook duration: ~2 hours

  • Issue is intermittent, the same build often succeeds on retry

Configuration

BuildConfiguration.xml

<bAllowUBA>true</bAllowUBA>
<bAllowUBAShaderCompile>true</bAllowUBAShaderCompile>

DefaultEngine.ini

[Uba.Provider.Horde]
Enabled=true
ServerUrl="<https://can-horde-server>..."
Pool=BuildMachine
Cluster=default
MaxCores=2048

[UbaController]
Providers=Uba.Provider.Horde
bReuseProcesses=True

[DevOptions.Shaders]
bAllowCompilingThroughWorkers=True
bAllowAsynchronousShaderCompiling=True
bAllowDistributedShaderCompiling=True
bUseXGEController=False
bUseHordeShaderCompiler=True

MaxShaderJobBatchSize=16
WorkerTimeToLive=600
NumUnusedShaderCompilingThreads=2

Observed Behavior

At the beginning of the cook, UBA + Horde appears to initialize correctly:

LogUbaHorde: UBA/Horde Configuration:
  Server: <https://can-horde-server>...
  Pool: BuildMachine

We also see:

LogShaderCompilers: Display: Using UBA Controller for shader compilation
LogUbaController: Starting up UBA/Horde connection for session ...

However, later during the cook:

LogUbaController: Shutting down UBA/Horde connection

Then shortly after:

LogUbaController: Starting up UBA/Horde connection for session ...
LogUbaHorde: Warning: Getting Horde server URL failed [Source: Not found]
LogUbaHorde: Error: Failed to create HttpClient for UbaAgent

At this point the cook fails.

What we observed

  • The same build often succeeds on retry

  • The failure happens after a successful initial Horde session

  • The error occurs during re-initialization of UBA/Horde

  • C++ compilation via UBA/Horde is stable

  • Issue appears only in shader compilation during cook

  • Logs suggest the ServerUrl becomes unavailable on second initialization

What we tried

  • Fixed INI parsing issues (quotes around ServerUrl)

  • Explicit [UbaController] Providers=...

  • Enabled bUseHordeShaderCompiler=True

  • Increased WorkerTimeToLive

  • Enabled bReuseProcesses=True

  • Cleared Saved/Intermediate configs

  • Verified Horde pool and agents are valid

Suspicions

We suspect one (or a combination) of the following:

  1. UbaController lifecycle issue in UE 5.6

    • Config not reloaded on second initialization

  2. Version mismatch

    • UE 5.6 client vs Horde/UBA agents 5.7

  3. Shader compilation path being less stable than UBT path

  4. Session teardown/recreation during long cooks causing loss of Horde config

Questions

  1. Is UE 5.6 expected to be stable for UBA + Horde shader compilation during cook?

  2. Are client/agent version mismatches (5.6 vs 5.7) known to cause this kind of issue?

  3. Should the Horde config be reloaded on every UbaController initialization?

  4. Is there a known fix or patch for this in 5.6?

  5. Are there recommended settings to prevent UbaController teardown during long cooks?

Any guidance would be greatly appreciated, especially if this is a known issue or something already fixed in later versions. Happy to provide full logs or test specific changes if needed.

Thanks!
Nathaël Goudreau