Our gauntlet started reporting failed test runs with error “Engine initialization failed” because it can’t find the string “Engine is initialized” in the log it parses. I dug in a little bit and it looks like Gauntlet uses a circular buffer to read the log that defaults to 1024 lines- which has suddenly become too few lines. I see there’s a command line param Gauntlet.LogBufferLineCapacity we can set to increase this, but does it make sense that tests should fail because of a commandline param not specifying enough log lines to check? Should we just have gauntlet check the entire log by default?
The circular buffer is supposed to be used only during live parsing of the log.
The parsing done for the initialization validation should be done through the log file itself. If it still using the circular buffer it means that it fails to save the log file or the log file was moved before the initialization validation has happened.
Looks like the issue was known and fixed but it did not make it to 5.7.
Here a patch to fix that:
Change 46418278 by Jerome.Delattre on 2025/10/01 09:29:54
Update the process log file path when written to file/copy
to still reference a valid log file path
Affected files ...
... //UE5/Main/Engine/Source/Programs/AutomationTool/Gauntlet/Framework/Base/Gauntlet.TargetDeviceDesktopCommon.cs#25 edit
Differences ...
==== //UE5/Main/Engine/Source/Programs/AutomationTool/Gauntlet/Framework/Base/Gauntlet.TargetDeviceDesktopCommon.cs#25 (text) ====
[Content removed]8 @@
return new LogFileReader(ProcessLogFile);
}
+ Log.Warning(KnownLogEvents.Horde_BuildHealth_Ignore, "Log file '{filepath}' is missing. The buffer will be used instead but will most likely lack the beginning of the file.", ProcessLogFile);
+
return ProcessLogOutput.GetReader();
}
[Content removed]16 @@
{
ProcessUtils.CheckProcessLogReachedSizeLimit(new FileReference(ProcessLogFile));
File.Copy(ProcessLogFile, FilePath, true);
+ if (HasExited)
+ {
+ // Follow the new location
+ // SaveRoleArtifacts may clean device cache folder prior parsing the log file
+ ProcessLogFile = FilePath;
+ }
}
else
{
- Log.Warning("Log file '{filepath}' is missing at the time of making a copy. The buffer will be used instead but will most likely lack the beginning of the file.", ProcessLogFile);
+ Log.Warning(KnownLogEvents.Horde_BuildHealth_Ignore, "Log file '{filepath}' is missing at the time of making a copy. The buffer will be used instead but will most likely lack the beginning of the file.", ProcessLogFile);
StreamWriter Writer = ProcessUtils.CreateWriterForProcessLog(FilePath, CommandLine);
foreach(string Line in ProcessLogOutput)
{
The critical part being the HasExited condition and the update of the file path.