Horde: file list create and read from different locations

Development Pipeline & Plugins
, ,
1

Hello

The root of my Perforce stream contains a subfolder “UE“ so when my Horde agents sync, everything goes to <WKS_ROOT>/UE/Engine and /UE/MyProject

In the myproject.stream.json config file (Horde server side), I’ve added “enginePath”: “UE/Engine” to be sure runuat will be run from the proper location.

It works… but there is problem when saving file list at the end of the process.

Here is the output I have in Horde:

Saving file list to D:\HordeAgent\P4\Main\Sync\Engine\Saved\BuildGraph\MyProject Update Version Files Win64\Tag-MyProject Update Version Files Win64.xml

(…)

Reading local file list from D:\HordeAgent\P4\Main\Sync\UE\Engine\Saved\BuildGraph\MyProject Update Version Files Win64\Tag-MyProject Update Version Files Win64.xml

Exception while executing step: System.IO.DirectoryNotFoundException: Could not find a part of the path ‘D:\HordeAgent\P4\Main\Sync\UE\Engine\Saved\BuildGraph\MyProject Update Version Files Win64\Tag-MyProject Update Version Files Win64.xml’.

We can see the file is not created at the same location that it is read later in the process (“UE” subfolder under “Sync”)

In Horde source code, we see that the root path depends on the env variable uebp_LOCAL_ROOT which is hardcoded and it does not take into account the value “enginePath” specified in the *.stream.json config file.

The workaround for me was to create a virtual stream in perforce that remaps UE/Engine to Engine and UE/MyProject to MyProject. It works… but there’s many side effects when using UGS, because UGS does not use this virtual stream and when it sends requests to Horde, it needs to pass the matching stream name.

The solution would be to remove this “UE” subfolder in Perforce but such a move is not harmless.

I am modifying UGS to add streamMapping feature to map ugs stream with horde stream. It’s working but not ideal as you can imagine.

Could you confirm the issue in Horde?

Thanks a lot

Mathieu

[Attachment Removed]

2

Steps to Reproduce

  1. Create a mainline stream in Perforce, for example: //depot/dev/main
  2. Submit engine and project under a subfolder “UE”. In Perforce you should have: //depot/dev/main/UE/Engine and //depot/dev/main/UE/MyProject
  3. Because Horde does not like no-standard structure in Perforce, add “enginePath”: “UE/Engine” in the myProject.stream.json config file
  4. Launch any buildgraph from Horde
  5. Note the step “Saving file list” does not create the file at the proper location (it does not take into account the “enginePath” value and try to create it to //depot/dev/main/Engine/Saved folder (no “UE” subfolder in the path)
  6. The process crashes at the end because the process try to read the file under the UE folder
    [Attachment Removed]
3

I deployed a new docker image with the fix and it works like a charm.

Thanks a lot Austin!

[Attachment Removed]

4

In case it has not already been noticed, an issue remained when the build process was distributed across multiple agents.

The Manifest.xml file is written by agent A but read by agent B from another location.

In Engine/Source/Programs/Horde/Drivers/JobDriver/Execution/JobExecutor.cs:

// line 1231: 
TempStorageBlockManifest manifest = await TempStorage.RetrieveBlockAsync(HordeClient, JobId, step.StepId, inputStorageBlock.NodeName, inputStorageBlock.OutputName, workspaceDir, manifestDir, logger, cancellationToken);
 
// line 1249:
else if (!inputFile.Equals(source.File) && !TempStorage.IsDuplicateBuildProduct(inputFile.ToFileReference(workspaceDir)))

In both cases, workspaceDir must be replaced by unrealRootDir.

[Attachment Removed]