How to set up the Automated Performance Testing to send and view results in Horde Performance Trends dashboard, and not having to look through it via the SummaryTable.html locally?

Development Pipeline & Plugins
,
21

That is correct - the more builds you run this on, the more dense the data is and longer the trend lines will get.

One thing that you could do, is attach the HTML Summary report as an artefact to the Horde step this executed from. That way, when you click on the dot it shows you all the horde runs associated with the data set. This only works if this APT run was sourced from Horde, otherwise it can’t link back to the steps.

I am exploring more support for non horde sourced builds, which really speaks to a lot of the challenges via the direct PRT emission that you may be facing - all the Horde Template-Job-Step-Stream context is missing. I’ll provide some more updates on that in the future, but this will require out of band patching of PerfReportTool and Horde Server/Dashboard (until I can get it into the official 5.8.X patching stream).

The whole Horde Perf Trend feature stems from the fact that reasoning about granular perf data (aka intra build) makes it difficult to introspect; you don’t know if it’s a build worth looking at. If, however, you have the “top of funnel” trendline, you can see that there’s a build worth diving into (and reviewing the detailed CSV data - not the summary table). Fundamentally, it’s about the information & diagnostic hierarchy.

Kind regards,

Julian

22

Just an update on my end. I am working on hardening the outside-of-horde emission path. Meaning that the dashboards & emission tool (PRT) will better support data sent from outside the APT/Horde build path.

The issues seen in the above thread are multivariate, but some of it is attributable to certain properties not being present in the dataset.

What has happened is that as the telemetry was not source from Horde (and with a csvbin containing key env data from horde), emissions would be incomplete and schematize wrong (which led to the above user having to manually add columns). From here, the dashboard would silently fail, and a confusing experience subsumes.

Once I get a target patch date I’ll reply here with the info. Thanks for the above details though, as it helped track down some of the use cases that we didn’t see internally around out-of-band telemetry emissions.

Kind regards,

Julian