BuildGraph support for <Agent> inside of <ForEach> inside of <Macro>

Development Pipeline & Plugins
, ,
1

Hi, it seems like BuildGraph doesn’t support this hierarchy:

<Macro Name="My Macro"> <ForEach Name="Target" Values="Client;Server"> <Agent Name="Test agent $(Target)" Type="test type"> <Node Name="Test node $(Target)"> <Log Message="TEST $(Platform) $(Target) "/> </Node> </Agent> </ForEach> </Macro>even though it should probably be valid. I’d like to generate a list of agents based on target/platform for parallelisation, separated out to a macro for XML cleanliness.

The error I get with this is `error: The element ‘ForEach’ in namespace ‘http://www.epicgames.com/BuildGraph’ has invalid child element ‘Agent’ in namespace ‘http://www.epicgames.com/BuildGraph’.`

I think I tracked it down to `CreateMacroBodyType` in `BgSchema.cs`. This function adds the ForEach element to macros, but only with inner content type `NodeBody`:

macroChoice.Items.Add(CreateForEachElement(ScriptSchemaStandardType.NodeBody));and obviously “Agent” isn’t a valid element within “Node”, so it fails.

I tried changing that line to “MacroBody” so that it’s recursive, but that also didn’t work. It failed with “error: Unexpected element type ‘Agent’”.

2

Hey there,

Thanks for reaching out regarding this use case. I’ve spoken with the team, and there are some challenges in implementing this in Build Graph 1. Build Graph 2 is quite experimental at the moment, but this would be the area of investigation & application for such a use case. As an example of a BuildGraph2 script, please refer to snippet below:

`using System.Collections.Generic;
using System.Threading.Tasks;
using AutomationTool;
using EpicGames.BuildGraph;
using EpicGames.BuildGraph.Expressions;
using EpicGames.Core;
using UnrealBuildTool;
using static AutomationTool.Tasks.StandardTasks;

namespace Ulysses.Automation.Graphs;

class ExampleGraph : BgGraphBuilder
{
public override BgGraph CreateGraph(BgEnvironment env)
{
BgAgent agent1 = new BgAgent(“agent 1”, “CompileWin64Incremental”);
BgAgent agent2 = new BgAgent(“agent 2”, “CompileWin64Incremental2”);
BgAgent agent3 = new BgAgent(“agent 3”, “CompileWin64Incremental3”);

BgNode<(BgFileSet, BgFileSet)> compileEditorNode = agent1.AddNode(x => CompileEditorAsync());
BgNode listBinariesNode = agent2.AddNode(x => PrintRequirementsAsync(x, “binaries”, compileEditorNode.Output.Item1)).Requires(compileEditorNode.Output.Item1);
BgNode listSymbolsNode = agent3.AddNode(x => PrintRequirementsAsync(x, “symbols”, compileEditorNode.Output.Item2)).Requires(compileEditorNode.Output.Item2);

return new BgGraph(new List { compileEditorNode, listBinariesNode, listSymbolsNode }, new List { new(“Aggregate”, listBinariesNode, listSymbolsNode) });
}

private static async Task<(BgFileSet, BgFileSet)> CompileEditorAsync()
{
FileSet allFiles = await CompileAsync($“EngineTestEditor”, UnrealTargetPlatform.Win64, UnrealTargetConfiguration.Development);

return (allFiles.Except(“.pdb"), allFiles.Filter(".pdb”));
}

[BgNodeName(“PrintRequirementsAsync {name}”)]
[System.Diagnostics.CodeAnalysis.SuppressMessage(“Style”, “IDE0060:Remove unused parameter”, Justification = “”)]
private static Task PrintRequirementsAsync(BgContext context, string name, BgFileSet files)
{
return Task.CompletedTask;
}
}`This is a script referenced from a different EPS [Content removed]

You can launch such a script like this:

  • RunUAT.bat BuildGraph -Target=“agent 1” -Class=ExampleGraph"

In the above system you should be able to achieve that you’re looking for in the interim.

Kind regards,

Julian

3

Oh awesome I didn’t even know about BuildGraph2! That’s good to know, thanks. Two follow up questions about it:

  1. Is there any rough roadmap for BuildGraph2 (that you’re able to share)? When would it be out of experimental?
  2. Are there any plans to mix V1 and V2, being able to call out to a V2 style graph from V1 (via a custom task or directly from XML)? So for my specific macro example, I could put that in C# in the V2 style but without having to convert my entire BuildGraph to C#, which is a non-trivial amount of effort (and might not even be desired in some cases).
4

Hey there!

As it’s still in quite an experimental state I can’t really speak with certainty to any timelines or interoperability between BG1 & BG2.

For what it’s worth:

  • BG2 is the direction we are now moving in for some new utilities used in our build farm context.
  • Interoperability would present a complex challenge based on how we resolve the graphs. I could see people working around this by having a separate template job (authored in V2) that is just called via buildgraph (authored in V1). If you however want to have the V2 fragment perform structural changes within the context of a BG1 job, this may be quite difficult to support. I can still raise this use case to the team as they consider BG2 plans - built on the premise of buildgraph conversion as there may be better ways to do this than hybrid interoperability.

Kind regards,

Julian