[Horde] Error while submitting new UnrealGameSync binaries to Perforce via Horde task - BuildUgs.xml

Development Pipeline & Plugins
, ,
1

I’ve added a config to check for changes in the Ugs source code path, that would build / deploy a new UGS versions.

So far so good, but strangely, the parameters passed in the project.stream.json files are either not applied, or just aren’t meant to be used the same way we can use them with the editor buildgraph.

As mentionned in the repro steps, I’ve added the params in the project.stream.json as follow :

  • Set the required parameters asked in the Submission Node : In example, here is what’s in the project.stream.json file :

“id”: “UGS-build”,

“name”: “UGS Build”,

“description”: “Build and Deploy UGS if changes are detected in the sources”,

“Pool”: “UGS”,

“showUgsBadges”: false,

“arguments”: [

“-target=Publish Live With Zip”,

“-Script=Engine/Source/Programs/UnrealGameSync/BuildUGS.xml”,

“-set:InformationalVersion=5.5.$(Change)”,

“-set:SubmitToBranch=//DestinationBranch/…”,

“-set:SubmitToStream=//project/Stream”,

  • “-set:HordeSettings=//PathTo/HordeSettings.json”

The perforce workspace is created, yet not hooked to the stream that could allow it to submit the newly build binaries, giving the following warning : warning: cannot submit from non-stream client to every files. I can confirm that the created workspace isn’t hooked on anything.

I compared with what’s found in BuildEditorAndTools.xml…

<Node Name=“Submit To Perforce For UGS” Requires=“Stage for UGS”>

<Property Name=“ArchivePerforceDir” Value=“$(RootDir)\LocalBuilds\ArchiveForUGS-Perforce”/>

<Property Name=“ArchiveFile” Value=“$(ArchivePerforceDir)\$(EscapedBranch)-$(ArchiveName).zip”/>

<Zip FromDir=“$(ArchiveStagingDir)” ZipFile=“$(ArchiveFile)”/>

<Warning Message=“The ‘ArchiveStream’ argument is not set. Binaries will not be submitted.” If=“‘$(ArchiveStream)’ == ‘’”/>

<Property Name=“SubmitClient” Value=“$(COMPUTERNAME)_ArchiveForUGS”/>

<Property Name=“SubmitClient” Value=“$(P4CLIENT)_ArchiveForUGS” If=“‘$(P4CLIENT)’ != ‘’”/>

<Submit Description=“[CL $(CodeChange)] Updated binaries” Files=“$(ArchiveFile)” FileType=“binary+FS32” Workspace=“$(SubmitClient)” Stream=“$(ArchiveStream)” RootDir=“$(ArchivePerforceDir)” Force=“$(ForceSubmit)” If=“‘$(ArchiveStream)’ != ‘’”/>

</Node>

The stream we set for Stream=“$(ArchiveStream)” will be used by the workspace created in the process… Wouldn’t it be the same for BuildUGS.xml?

2

Steps to Reproduce

  • Set a configuration for building / deploying UGS via Horde using BuildUgs.xml (which is provided in UGS source code path)
  • Set the required parameters asked in the Submission Node : In example, here is what’s in the project.stream.json file :

“id”: “UGS-build”,

“name”: “UGS Build”,

“description”: “Build and Deploy UGS if changes are detected in the sources”,

“Pool”: “UGS”,

“showUgsBadges”: false,

“arguments”: [

“-target=Publish Live With Zip”,

“-Script=Engine/Source/Programs/UnrealGameSync/BuildUGS.xml”,

“-set:InformationalVersion=5.5.$(Change)”,

“-set:SubmitToBranch=//DestinationBranch/…”,

“-set:SubmitToStream=//project/Stream”,

  • “-set:HordeSettings=//PathTo/HordeSettings.json”
    • For clarification, the SubmitToBranch= param is poiting to where I want the new ugs binaries to be submitted on Perforce. SubmitToStream, however, is a bit confusing. I was expecting the Perforce Client created by the process to be hooked to that stream.

This is based of BuildUgs.xml requirements from the Node here :

<Node Name=“Publish Live” Requires=“Build UnrealGameSync”>

<Property Name=“ReleaseBranch” Value=“”/>

<Property Name=“ReleaseBranch” Value=“$(SubmitToBranch)/Release” If=“‘$(SubmitToBranch)’ != ‘’”/>

<Property Name=“SubmitClient” Value=“$(COMPUTERNAME)_UgsUpdater”/>

<Property Name=“SubmitClient” Value=“$(P4CLIENT)_UgsUpdater” If=“‘$(P4CLIENT)’ != ‘’”/>

<Submit Description=“Updating UGS from CL $(Change)” Workspace=“$(SubmitClient)” Files=“$(UgsPublishDir)/…” RootDir=“$(UgsOutputDir)” Stream=“$(SubmitToStream)” Branch=“$(ReleaseBranch)”/>

</Node>

<Node Name=“Publish Live Zip” Requires=“Build UnrealGameSync Zip”>

<Copy From=“$(UgsZipPublishDir)/Ugs.zip” To=“$(UgsZipPublishDir)/Release.zip”/>

<Property Name=“SubmitClient” Value=“$(COMPUTERNAME)_UgsUpdater”/>

<Property Name=“SubmitClient” Value=“$(P4CLIENT)_UgsUpdater” If=“‘$(P4CLIENT)’ != ‘’”/>

<Submit Description=“Updating UGS from CL $(Change) (Zip)” Workspace=“$(SubmitClient)” Files=“$(UgsZipPublishDir)/Release.zip” RootDir=“$(UgsZipOutputDir)” Stream=“$(SubmitToStream)” Branch=“$(SubmitToBranch)”/>

</Node>

The client that would submit the files is created… but since it’s not hooked on the Stream=“$(SubmitToStream)” param, i’m getting : warning: cannot submit from non-stream client on every files.

3

Hey there Mathieu,

Sorry for the delay, as the team has been away at a conference.

I do not see us setting the values at all of SubmitToStream internally. As a matter of fact, our arguments are quite concise:

  • “-Script=Engine/Source/Programs/UnrealGameSync/BuildUGS.xml”,
  • “-set:SubmitToBranch=PATH_TO_PUBLISH_DEPOT/UnrealGameSync/bin”,
  • “-set:HordeSettings=PATH_TO_TOOL_UPLOAD.json”,
  • “-set:HordeSettingsMac=PATH_TO_TOOL_UPLOAD.json”,
  • “-set:HordeSettingsLinux=PATH_TO_TOOL_UPLOAD.json”

Just reviewing the code a bit here, have you tried without setting the SubmitToStream param? It doesn’t look strictly required within the underlying SubmitTask and I suspect that’s where some of the issues could be coming in.

Kind regards,

Julian

4

Hey there Mathieu,

Just to confirm, have you tried submitting just to the branch, without the additional SubmitToStream param? Any luck on this, or are you still seeing issues?

Kind regards,

Julian

5

Yeah I am now trying to deploy it through UGS directly instead of SubmitToStream… Of course, not much success here.

6

I did try, without success. SubmitToBranch has the same behaviour… and nothing gets submitted in the end.

7

I’m curious whether a pre-existing workspace has been created for the particular submission machine that’s now causing issues.

I wonder if you could add some debug printouts to the SubmitTask.cs code, so that we can get a sense of whether it’s trying to create a new workspace or not. If the SubmitToBranch has the same behaviour - that is “warning: cannot submit from non-stream client” - it seems that the workspace and target are mismatched here?

`P4Connection submitP4 = CommandUtils.P4;
if (_parameters.Workspace != null)
{
// Create a brand new workspace
P4ClientInfo client = new P4ClientInfo();
client.Owner = CommandUtils.P4Env.User;
client.Host = Unreal.MachineName;
client.RootPath = parameters.RootDir.FullName ?? Unreal.RootDirectory.FullName;
client.Name = $"{parameters.Workspace}{Regex.Replace(client.Host, “[^a-zA-Z0-9]”, “-”)}{ContentHash.MD5((CommandUtils.P4Env.ServerAndPort ?? “”).ToUpperInvariant())}“;
client.Options = P4ClientOption.NoAllWrite | P4ClientOption.Clobber | P4ClientOption.NoCompress | P4ClientOption.Unlocked | P4ClientOption.NoModTime | P4ClientOption.RmDir;
client.LineEnd = P4LineEnd.Local;
if (!String.IsNullOrEmpty(_parameters.Branch))
{
client.View.Add(new KeyValuePair<string, string>($”{_parameters.Branch}/…“, $”/…"));
}
else
{
client.Stream = _parameters.Stream ?? CommandUtils.P4Env.Branch;
}
CommandUtils.P4.CreateClient(client, AllowSpew: _parameters.P4Verbose);

// Create a new connection for it
submitP4 = new P4Connection(client.Owner, client.Name);
}`If you could attach the failed log, that would be helpful as well in tracking down exactly what code path is being executed here (if you add debug output).

Julian