We are requesting an opt-in improvement to Horde’s preflight auto-submit behavior.
Currently, a preflight configured for auto-submit is not submitted when any step completes with warnings. This requires manual intervention even when every warning is already represented by an existing open Horde issue.
The proposed AutoSubmitPreflightWithKnownWarnings job option changes this behavior when explicitly enabled. Horde will:
Generate issue fingerprints for each warning step.
Compare them with open issue spans for the same stream, template, and node.
Allow auto-submit when every generated fingerprint matches an existing open issue.
Block auto-submit when any warning represents a new issue.
Fail closed if validation is unavailable, fails, or required configuration cannot be resolved.
Record the blocking reason for notification consumers.
A warning step that produces no issue fingerprints does not represent a newly detected issue and therefore does not block auto-submit.
The default behavior remains unchanged unless the new option is enabled. This would benefit projects using Horde preflight auto-submit and issue fingerprinting by reducing unnecessary manual submissions without allowing newly detected warnings through unnoticed.
We would appreciate feedback on whether matching warning fingerprints against open issue spans is the preferred definition of a known warning and whether warning steps without fingerprints should instead block auto-submit.
Configure a Horde template with preflight auto-submit enabled.
Run the template on a submitted change that produces a deterministic warning.
Allow Horde to create an issue for that warning.
Run a preflight containing an unrelated change that produces the same warning fingerprint.
Wait for the preflight to complete with a warning outcome.
Change does not submit due to no fault of the change under preflight
Current Result: The preflight is not auto-submitted, even though the warning matches an existing open Horde issue.
Expected Result: With the proposed option enabled, Horde recognizes the warning as known and auto-submits the preflight. New warning fingerprints must continue to block auto-submit.
No template game project is required because this behavior is within Horde’s server-side job and issue-processing workflow.
Thank you for creating the pull request, including a detailed write-up, and adding the tests, too!
The pull request has been picked up by our UE GitHub PR review process, so it will be routed to the Horde team for review and feedback (via the PR).
On your questions, matching warning fingerprints against open issue spans does make sense as a definition of a known warning.
This is already how warnings are ingested and attached to spans in the system:
“Build errors are ingested and attached to spans via their fingerprint. Fingerprints provide a programmatic description of the error, and how it should be matched with other errors.”
