UbaController Plugin Shader Compliation error with Unreal Build Accelerator

Development Pipeline & Plugins
, ,
1

Attempting to enable distributed shader compilation using the UbaController plugin with Horde/UBA fails with: `Failed to detour CreateProcessW (The function referenced is too small to be detoured.)`

I noticed Epic’s main branch appears to have UbaController changes since my current version of 5.7 that may mitigate this issue but wondering if it is the intended fix and available in 5.8 or if there is an easy changelist to grab?

Full logs:

```

LogShaderCompilers: Display: Using UBA Controller for shader compilation

LogUbaController: Display: Starting up UBA/Horde connection for session UbaController_0274E70147971BBDA7261390FD4E0CEE

LogUbaController: Display: Created UBA storage server: RootDir=C:/Users/xxxx/AppData/Local/Temp/UbaControllerStorageDir/0

LogUbaController: Display: Failed to detour CreateProcessW (The function referenced is too small to be detoured.)

LogUbaController: Display: Failed to detour CreateProcessW (The function referenced is too small to be detoured.)

LogUbaController: Display: ERROR: Process 15756 0.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassPSFNoLightMapPolicy-0.in (G:\Engine\Binaries\Win64\ShaderCompileWorker.exe) not active but did not get exit message. Received 1 messages (GetExitCodeProcess returned 0x9)

LogUbaController: Display: ERROR: Process 15456 1.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassVSFNoLightMapPolicy-0.in (G:\Engine\Binaries\Win64\ShaderCompileWorker.exe) not active but did not get exit message. Received 1 messages (GetExitCodeProcess returned 0x9)

LogUbaController: Display: Distributed job output file [C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/0/0.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassPSFNoLightMapPolicy-0.out] is invalid or does not exist

- Directory “C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/0” exists and contains 0 file(s)

LogUbaController: Save snapshot of UBA trace: G:/Game/Saved/UbaController/UbaController.MultiprocessId-0.Session-0.uba

LogUbaController: Display: Distributed job output file [C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/1/1.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassVSFNoLightMapPolicy-0.out] is invalid or does not exist

- Directory “C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/1” exists and contains 0 file(s)

LogUbaController: Save snapshot of UBA trace: G:/Game/Saved/UbaController/UbaController.MultiprocessId-0.Session-0.uba

LogShaderCompilers: Display: Rescheduling shader compilation to run locally after distributed job failed: C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/1/1.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassVSFNoLightMapPolicy-0.out

LogShaderCompilers: Display: Rescheduling shader compilation to run locally after distributed job failed: C:/Users/xxxx/AppData/Local/Temp/UbaControllerWorkingDir/B77452334B598BB49FC681AFC02D1405/0/0.uba.FMatExpressionPreview-MaterialExpressionConstant3Vector_0_c51ba1cf8c3c820f-Default-FLocalVertexFactory-TBasePassPSFNoLightMapPolicy-0.out

```

[Attachment Removed]

2

Steps to Reproduce

  1. Enable `UbaController` plugin in .uproject file
  2. Configure UBA with Horde and UbaController for project in a Engine ini file
  3. Launch editor and do something that triggers a shader compilation (i.e open a material)
  4. Notice the errors in the logs from LogUbaController
    [Attachment Removed]
3

This is very likely something else running on this machine hooking same functions… things like

- Antivirus / EDR (Defender ATP, CrowdStrike, SentinelOne, Carbon Black, Sophos)

- Sandboxing / app-virtualization shims

- Another Detours-based tool injected first

Talk to IT or whoever is in charge of installing these things on the machines and see what you find. If you can’t figure it out we can explore other approaches (that are more fragile)

[Attachment Removed]

4

I just checked in a changelist that hopefully will report reason it can’t detour CreateProcessW.. it will be mirrored to github over night I assume:

[UBA]

* Added code to try to explain reason when not being able to detour

[Attachment Removed]

5

(You will need to compile the binaries yourself, no new binaries have been queued yet)

[Attachment Removed]

6

New binaries are in from earlier today

[Attachment Removed]

7

Apologies for the delay in getting back and thank you for the information and providing a change to help identify what is going on.

After some more investigation on my end, I had the “Microsoft Child Process Debuggin Power Tool” VS extension enabled that was hooking into the same functions that UbaController was trying to detour. Disabling this extension allowed the shaders to compile successfully using UBA.

Thank you and hopefully this info is helpful to know as well!

[Attachment Removed]

8

yes, that extension hooks CreateProcess etc and make uba fail in detouring child processes

[Attachment Removed]