How to run BuildGraph with a minimal setup

Development Pipeline & Plugins
, ,
1

(This is a translation of a [Japanese [Content removed] by Kadowaki Yuki.)

I am thinking of preparing various automation jobs on Horde, including P4 operations. These are administrative jobs that are completely unrelated to Unreal Engine. For this reason, I created a Perforce stream that was apart from for any UE project, and placed files including batch files and a BuildGraph script for Horde execution. However, since running BuildGraph naturally requires RunUAT and etc., those files alone are not sufficient and the Horde jobs fail.

That said, making full use of the Engine for administrative or miscellaneous jobs appears unnecessary, particularly considering the disk space usage of the Horde Agent client workspace.

Jenkins commonly runs such miscellaneous or administrative jobs, and I would like to complete these workflows entirely within Horde, as well. For this, what files and directories are actually required?

Currently, I have tried the scope as follows, but I am seeing an error where uebp_CodeCL ends up being set to 0.

◆ Perforce Stream Paths

share …
import Engine/Build/BatchFiles/RunUAT.bat //EngineDepot/5.7.1/Engine/Build/BatchFiles/RunUAT.bat
import Engine/Build/BatchFiles/BuildUAT.bat //EngineDepot/5.7.1/Engine/Build/BatchFiles/BuildUAT.bat
import Engine/Build/BatchFiles/BuildUBT.bat //EngineDepot/5.7.1/Engine/Build/BatchFiles/BuildUBT.bat
import Engine/Build/BatchFiles/DotnetDepends.bat //EngineDepot/5.7.1/Engine/Build/BatchFiles/DotnetDepends.bat
import Engine/Build/BatchFiles/GetDotnetPath.bat //EngineDepot/5.7.1/Engine/Build/BatchFiles/GetDotnetPath.bat
import Engine/Build/Build.version //EngineDepot/5.7.1/Engine/Build/Build.version
import Engine/Config/BaseEngine.ini //EngineDepot/5.7.1/Engine/Config/BaseEngine.ini
import Engine/Source/Programs/AutomationTool/… //EngineDepot/5.7.1/Engine/Source/Programs/AutomationTool/…
import Engine/Source/Programs/AutomationToolLauncher/… //EngineDepot/5.7.1/Engine/Source/Programs/AutomationToolLauncher/…
import Engine/Source/Programs/UnrealBuildTool/… //EngineDepot/5.7.1/Engine/Source/Programs/UnrealBuildTool/…
import Engine/Source/Programs/Shared/… //EngineDepot/5.7.1/Engine/Source/Programs/Shared/…
import Engine/Binaries/DotNET/… //EngineDepot/5.7.1/Engine/Binaries/DotNET/…
import Engine/Binaries/ThirdParty/fastJSON/… //EngineDepot/5.7.1/Engine/Binaries/ThirdParty/fastJSON/…
import Engine/Binaries/ThirdParty/DotNet/… //EngineDepot/5.7.1/Engine/Binaries/ThirdParty/DotNet/…
import Engine/Config/… //EngineDepot/5.7.1/Engine/Config/…
import Engine/Build/Graph/… //EngineDepot/5.7.1/Engine/Build/Graph/…
import Engine/ThirdParty/IOS/… //EngineDepot/5.7.1/Engine/ThirdParty/IOS/…

◆ Error

P4Environment.CodeChangelist has not been initialized but is requested. Set uebp_CodeCL env var or run UAT with -P4 to automatically detect changelist.

◆ Actions Taken

To address this error, I tried the following, but uebp_CodeCL still remained 0:

- Placed a dummy .h file at the root of the Perforce stream (as a workaround for DetectCurrentCodeCL).

- Specified “uebp_CodeCL”: “$(uebp_CL)” in the environment settings under the agentType in *.stream.json.

I guess this may be occurring because certain files that should be under the Engine directory are missing. I’d appreciate it if you could advise me about the issue.

Thank you.

[Attachment Removed]

2

Hello,

This is the client spec I have

share .editorconfig
share Default.uprojectdirs
share GenerateProjectFiles.*
share RunUAT.*
share Engine/Binaries/DotNET/OidcToken/...
share Engine/Binaries/DotNET/Ionic.Zip.Reduced.*
share Engine/Binaries/DotNET/EpicGames.Perforce.Native/...
share Engine/Binaries/ThirdParty/DotNet/...
share Engine/Binaries/ThirdParty/fastJSON/...
share Engine/Binaries/ThirdParty/ICU/...
share Engine/Binaries/ThirdParty/Ionic/...
share Engine/Binaries/ThirdParty/VisualStudio/...
share Engine/Build/*
share Engine/Build/Graph/...
share Engine/Build/Horde/...
share Engine/Build/BatchFiles/...
share Engine/Build/Windows/...
share Engine/Config/...
share Engine/Extras/P4VUtils/...
share Engine/Extras/UnsupportedTools/StructuredLogging/...
share Engine/Extras/UnrealVS/...
share Engine/Extras/ushell/...
share Engine/Source/Developer/Horde/...
share Engine/Source/Programs/*
share Engine/Source/Programs/Shared/...
share Engine/Source/Programs/Horde/...
share Engine/Source/Programs/OidcToken/...
share Engine/Source/Programs/UnrealBuildTool/...
share Engine/Source/Programs/UnrealBuildTool.Tests/...
share Engine/Source/Programs/UnrealGameSync/...
share Engine/Source/Programs/AutomationTool/*
share Engine/Source/Programs/AutomationTool/AutomationUtils/...
share Engine/Source/Programs/AutomationTool/BuildGraph/...
share Engine/Source/Programs/AutomationTool/Localization/...
share Engine/Source/Programs/AutomationTool/Properties/...
share Engine/Source/Programs/AutomationTool.Tests/...
share Engine/Source/Programs/AutomationToolLauncher/...
share Engine/Source/Programs/Unsync/...
share Engine/Source/Programs/UnrealCloudDDC/...
share Engine/Source/Programs/UnrealToolbox/...
share Engine/Source/Programs/UnrealVS/...
share Engine/Source/Runtime/Core/Public/HAL/PreprocessorHelpers.h
share Engine/Source/Runtime/Launch/Resources/Version.h
share Engine/Source/Runtime/Launch/Resources/Windows/resource.h
share Engine/Source/ThirdParty/BLAKE3/...
share Engine/Source/ThirdParty/CLI11/...
share Engine/Source/ThirdParty/fmt/...
share Engine/Source/ThirdParty/http_parser/...
share Engine/Source/ThirdParty/LibreSSL/...
share Engine/Source/ThirdParty/OpenSSL/*
share Engine/Source/ThirdParty/OpenSSL/1.1.1t/...
share Engine/Source/ThirdParty/Perforce/p4api-2023.2/...
share Engine/Source/ThirdParty/vcpkg/...
share Engine/Source/ThirdParty/WiX/...
share Engine/Source/ThirdParty/zstd/...
share Engine/Platforms/*/Config/...

How are you running UAT?

If you place a breakpoint in Engine\Source\Programs\AutomationTool\AutomationUtils\P4Environment.cs on this line if(String.IsNullOrEmpty(CodeChangelistString) && CommandUtils.P4CLRequired)

line 220 (in latest version) are you able to step through?

Matthew

[Attachment Removed]

3

[mention removed]​

(This is a translation of a Japanese post by Kadowaki Yuki.)

Thank you for your reply.

Also, thank you for sharing your Perforce Client Workspace view settings! I’ll try them later.

> How are you running UAT?

According to the Horde logs, UAT is run as follows:

Executing C:\WINDOWS\system32\cmd.exe /C “”<DriveLetter>:\<AgentPath>\Sandbox\<agentTypeName>\Sync\Engine\Build\BatchFiles\RunUAT.bat" BuildGraph -Script=BuildGraph/<BuildGraphName>.xml -HordeExport=<DriveLetter>:\<AgentPath>\Sandbox\<agentTypeName>\Sync\Engine\Saved\Horde\Exported.json -ListOnly -Target=“Job Test” -UTF8Output -BuildMachine

> If you place a breakpoint in Engine\Source\Programs\AutomationTool\AutomationUtils\P4Environment.cs on this line…

I’ll be trying this by running a debug session locally. However, it may take some time. I will follow up with an update.

Thanks.

[Attachment Removed]

4

[mention removed]​

(This is a translation of a Japanese post by Kadowaki Yuki.)

I ran a Horde job (packaged-build) for another case using the following View settings, but the same error occurred.

So, it seems that the hypothesis that missing files prevent uebp_CodeCL from being retrieved was wrong.

share ...
import Engine/... //EngineDepot/5.7.1/Engine/...
import projects/Lyra/... //EngineDepot/5.7.1/Samples/Games/Lyra/...
isolate projects/Lyra/Build/...

Also, I have just run the stepped through under a debugger. Then, uebp_CodeCL was successfully retrieved on the Horde Server in the debug environment. So I restarted the production Horde Server, and uebp_CodeCL could be retrieved correctly.

(The server had already been restarted as of Feb 2, but the issue was not resolved at that time.)

In the end, unfortunately, the root cause remains unclear. So it doesn’t seem like a very clean resolution…

[Attachment Removed]