I have been trying to set this up for the past 3 days, the automated testing framework results generate ok locally into a SummaryTable.html, but I cannot send them reliably to Horde. It seems like it was made with P4 in mind, and I am not using that.
My goal: I just want to run the RunLocalTests and see the results in the Horde Performance Trends dashboard, and not look through it via the SummaryTable.html.
I am using the RunLocalTest and then sending it to horde with:
dotnet “C:\Unreal5.8\Engine\Binaries\DotNET\CsvTools/PerfReportTool.dll” -csvdir “C:\Unreal5.8\GameName\Saved\Performance\SequenceTest\Perf\Win64\CSV” -reportType ClientPerf -reportxmlbasedir “C:\Unreal5.8\Engine\Plugins\Performance\AutomatedPerfTesting\Build\Scripts\PerfReport” -summaryTable KeyStats -noDetailedReports -csvTable -summaryTableDataToTelemetry -telemetryTestIdentity “GameName”
The horde receives it fine (seemingly), but the resulting performance trends dashboard only shows one datapoint.
root@horde:~# docker logs -f horde-server | grep “Created table”
[10:23:54 inf] Created table horde.state_performance_key_stats from schema with 49 columns
[10:23:54 inf] Created table horde.state_performance_key_stats from schema with 49 columns
[10:23:54 inf] Created table horde.state_performance_key_stats from schema with 49 columns
Below are me server.json and globals.json (using placeholder db names and pws, this is running in local isolated proxmox LXC debian instance anyway).
The PerfReportTool does not fill the stream value, so the dashboard was not viewable, the data was wrong, and I wrote a simple “data backfill” for the missing db values. But still, the values are wrong… overall, settings this up was a complete headache. Matt Oztalay hyped me with his enthusiasm, because it seemed like it is working right now with his local tests scripts and it gets sent to Horde automatically! IT DOES NOT.
Please, could you write a more comprehensive guide or give me a tip how to put this together. I just want to run the RunLocalTests and see the results in the Horde Performance Trends dashboard, and not look through it via the SummaryTable.html.
//server.json
{
“Horde”: {
“AuthMethod”: “Anonymous”,
“ConfigPath”: “globals.json”,
“Perforce”: [
],
“Telemetry”: [
{
“Type”: “Mongo”,
“RetainDays”: 1
}
],
“UseLocalPerforceEnv”: false,
“Plugins”: {
“Analytics”: {
“Enabled”: true,
“Sinks”: {
“Mongo”: { “Enabled”: true, “RetainDays”: 1 }
}
},
“StructuredAnalytics”: {
“Enabled”: true,
“BackendName”: “ClickHouse”,
“DisableStreamBasedAuth”: true
},
“PerformanceTrends”: {
“Enabled”: true,
“DisableStreamBasedAuth”: true
},
“ClickHouseTelemetry”: {
“Enabled”: true,
“ConnectionString”: “Host=horde-clickhouse;Port=8123;Database=my_database;Username=username;Password=password”,
“AutoCreateTables”: true
}
}
}
}
and globals.json
root@horde:~# cat globals.json
{
“version”: 2,
“include”: [
{ “path”: “$(HordeDir)/Defaults/default.global.json” }
],
“plugins”: {
“clickhousetelemetry”: {
“ConnectionString”: “Host=horde-clickhouse;Port=8123;Database=my_database;Username=username;Password=password”,
“AutoCreateTables”: true
},
“structuredanalytics”: {
“BackendName”: “ClickHouse”,
“DisableStreamBasedAuth”: true
},
“performancetrends”: {
“BackendName”: “ClickHouse”,
“DisableStreamBasedAuth”: true
},
“analytics”: {
“stores”: [
{
“id”: “engine”,
“include”: [
{ “path”: “$(HordeDir)/Defaults/default-metrics.telemetry.json” },
{ “path”: “$(HordeDir)/Defaults/default-views.telemetry.json” }
]
}
]
}
}
}
I created the required db manually in clickhouse:
docker exec -it horde-clickhouse clickhouse-client --user username --password password --query "
356 CREATE TABLE ingest.horde_state_performance_key_stats
357 (
358 _telemetry_store_id String,
359 _app_id Nullable(String),
360 _app_version Nullable(String),
361 _app_environment Nullable(String),
362 _session_id Nullable(String),
363 dynamic_resolution_percentage_max Int64,
364 csvid Nullable(String),
365 gp_utime_avg Float64,
366 physical_used_mb_max Float64,
…