Clarification on Using Macros and Includes in Horde CI Configuration

Development Platform & Builds
, ,
1

Hello,

We are currently configuring Horde as our CI solution for Unreal Engine and aiming to reduce redundancy by using macros and included configuration files. However, we are facing an issue where macro variables defined in our stream files are not properly substituted when included configurations are processed.

Specifically, we define macros (e.g., “ProjectPath”, “EditorTarget”) in our stream configuration file, and then include separate template configuration files (using the “include” directive). However, variables such as “$(EditorTarget)” remain unsubstituted and appear literally (e.g., “$(EditorTarget) Win64” instead of “RandomGameNameEditor Win64”).

Could you please provide guidance on:

  • The correct order or method for defining macros relative to the “include” directive to ensure proper substitution?
  • Best practices or examples for using macros alongside included template files to effectively reduce duplication in Horde configuration?

Example of our current configuration structure:

{

“macros”: [

{ “name”: “EditorTarget”, “value”: “RandomGameNameEditor” }

],

“include”: [

{ “path”: “../templates/base-templates/incremental-build-template.json” }

]

}

incremental-build-template.json:

“arguments”: [

“-Target=Incremental Build”,

“-set:UProjectPath=$(ProjectPath)”,

“-set:EditorTarget=$(EditorTarget)”

]

Thank you very much for your assistance!

2

Steps to Reproduce

3

Hi there,

After reviewing the code that parses the includes and applies the macros (confignode.cs:529).

It parses the includes recursively, depth-first, applying the macros contained in that JSON file after parsing each include, but only to the current file.

As the entire JSON file is parsed, the order of the includes and macro definitions has no impact on the order the macros are applied.

I am unsure if this is by design to limit the scope and effect of macros to only the current file, rather than affecting includes and what those includes include, or an oversight.

If you are building your own Horde images from source,

You could try changing this behaviour by moving the macro parsing, but not the expanding, to before the Include recursive parsing. (Note: I have not tried this)

(confignode.cs:560 - 566)

This would affect all config JSON files. For example, you could have macros in the project.json, and you can also have stream config directly in the project.json, and use includes within that json object as well. This would make it possible to reduce your boilerplate down to just a project file. The {ID: “stream name”, Path: “./streampath.stream.json”} from the code comments is there for backwards compatibility with old config files when including from a ProjectConfig object.

Without code changes, I’d recommend moving the templates::Arguments (Or anywhere that the macros are being used) out of your includes into the same document/scope as the macro definitions.

I’ll reassign this task to Epic (it may take a week or so before they get back to you, as Epic is currently on break).

This will either clarify the design intent or raise a feature request ticket, perhaps adding a scope to the macro definitions, or adding parameters/macros to the include definition, or if macros should apply to includes as-well.

Kind Regards

Keegan Gibson