I’m currently doing some investigation on the capabilities for testing in Horde, so I apologize in advance if there are parts of Horde that I may misunderstand.
Anyway, does horde support test parallelization across multiple horde agents that run on Win64? The closest I have seen to test parallelization is through Gauntlet’s “parallel” argument through something like this:
RunUnreal -project="Project.uproject" -build=editor -test=UE.EditorAutomation(RunTest=Group:Group1),UE.EditorAutomation(RunTest=Group:Group2),UE.EditorAutomation(RunTest=Group:Group3) -parallel=3This however runs 3 editor session on the local machine- this isn’t really ideal for my case of running tests in parallel. I believe you can’t set other Win64 machines as devices either (correct me if I’m wrong) so you can’t orchestrate running parallel tests on those machines either. I also assume the devices in Horde can only be console and mobile devices.
Does Epic currently run tests in parallel? How is it or can it be done?
Hi Christian,
Yes, this should be doable. In short, Horde is more or less a BuildGraph executor. It will attempt to resolve a build graph, and where jobs are unrelated, it will in fact go as wide as possible at the Agent level.
As an example, if you review the following buildgraph:
https://github.com/EpicGames/UnrealEngine/blob/5.5/Engine/Build/Graph/Tasks/BuildAndTestProject.xml#L4
You can see there are several nodes at “parallel” depth, that don’t end up having dependencies on each other (but instead, input dependencies). This will mean that the nodes will be executed across multiple Horde agents at once. The same is very much true for build & test.
So fundamentally, Horde doesn’t “know” about the details of the job per-se, and have specific support for Gauntlet parallelism. It’s more that the tasks are determined to be parallelizable in general.
I hope this clears up the fundamentals a bit - the short answer is “if you can author your build graph in a way where you N sperate nodes executing different tests, you’ll get parallelism”.
Kind regards,
Julian
Hey Christian,
For simplicity, I’d think of this at the agent level:
I’ve cleaned up my original post because I think it’s a tiny bit misleading (Agent level is parallelism, Node is executed linearly within an agent & should be considered steps - apologies for this confusion).
Kind regards,
Julian
Ooh, neat! So let’s say we have some element <Agent> specified and I have 5 Horde agents that match that agent element. Now under the agent element, I have 3 RunUnreal nodes that all have the same input dependency (assuming it’s Requires=“$(TestPrerequisites)” for $(TestNodeName)), but not on each other. Would that result in 3 different agents running the required node then sequentially running the RunUnreal node?
Seems like this would be the path for getting 3 separate test groups to run on 3 different agents, i.e. parallelization. Group 1 goes to the first node, group 2 goes to second, etc.
Ah this makes a lot more sense to me with the examples you gave me! Super cool and I’m looking forward to messing with it, thanks Julian!