Talks And Demos: A Tech Artist's Guide to Automated Performance Testing | Unreal Fest Bali 2025

[ozzmeister002 (Matt Oztalay (He/Him))]

This guide should help you get up and running with your first automated performance tests using the newly-released (and very experimental) Automated Performance Testing plugin and framework.

https://dev.epicgames.com/community/learning/talks-and-demos/0zx9/unreal-engine-a-tech-artist-s-guide-to-automated-performance-testing-unreal-fest-bali-2025

[UsefulCode]

Hey Matt,
This thing works, and thank you for that. I tested it both locally and via Horde.
A few nuances when setting it up via Horde:

• Reports didn’t generate locally on agent until I added -set:ExtraGauntletCommonArgs=-SkipPerfReportServer -LocalReports.
  And judging by this comment in the code, the only option at the moment to generate a report is to do it locally on the agent.

/// /// Attempts to create the PerfReportServerImporter. /// This importer is currently internal to Epic and such /// this function will return null when not used internally. ///

• The Horde Agent must be run as a separate application, not as a service. Otherwise, the test won’t have access to the GPU context and won’t run.

I haven’t dug deep yet, but it seems there’s no way to add new custom tests without modifying the plugin code?

[ozzmeister002 (Matt Oztalay (He/Him))]

Hey @UsefulCode! Sorry I missed the note. Looks like there’s a few things to clean up in the report generation function. The way the arguments get set and passed around has a few double/triple negatives that can get confusing. I think for the time being your solution works well (and I’ll add it to the article), and we’ll see about tidying that up for a future release.

[eriktenstar]

Does this support setting which DataLayers should be part of the test somehow?

[UsefulCode]

Hey @eriktenstar
You can specify which Game Mode to use for your test. And within the Game Mode, you can add any logic specific to your test, including loading/unloading the required Data Layers.

[UsefulCode]

Automated Performance Testing plugin is excellent.
However, I personally really missed the functionality provided by the standard Perf Report Tool. I especially missed the ability to create comparative reports between multiple CSV captures. So, I created the Performance Analyzer plugin and use it in conjunction with the Automated Performance Testing plugin.

Unreal Engine | Profiling is tough. Performance Analyzer makes it easier.

[jilda]

Hello, @ozzmeister002, could you please help in this topic? I tried everything for 3 days, but the Performance Trends dashboard just refuses to display the data after running RunLocalTests

How to set up the Automated Performance Testing to send and view results in Horde Performance Trends dashboard, and not having to look through it via the SummaryTable.html locally?

I have been trying to set this up for the past 3 days, the automated testing framework results generate ok locally into a SummaryTable.html, but I cannot send them reliably to Horde. It seems like it was made with P4 in mind, and I am not using that. My goal: I just want to run the RunLocalTests and see the results in the Horde Performance Trends dashboard, and not look through it via the SummaryTable.html. I am using the RunLocalTest and then sending it to horde with: dotnet “C:\Unreal5.8\…

[ozzmeister002 (Matt Oztalay (He/Him))]

Just saw your ping on Slackers as well and I’ll post here for viz. The AutomatedPerfTestNode is only going to send results to Horde if Gauntlet sets IsBuildMachine, and if you’ve set HordeTelemetry to True in your BuildGraph arguments on your Horde job template.