Unreal will only report a maximum of 4 ensures via OnHandleSystemEnsure because of const uint32 MaxPreviousErrorsToTrack = 4;

Development Platform & Builds
, ,
1

Hey,

As stated in the title and detailed in the repro steps, the following code prevents more than 4 ensures report being broadcast via OnHandleSystemEnsure. This is a problem for plugins like Sentry which rely on OnHandleSystemEnsure to route ensures to their online services.

A quick fix is to just bump this number to something much higher but, I really fail to see what the goal is of this code and why it exists, can you please raise a bug to fix this? I shudder to think how many projects are loosing/delaying information about how their systems are operating because of this limit.

As far as I can tell it’s not even unique ensures, it’s just any ensure, even if it’s the same ensure 4 times.

Cheers.

if (GEnsuresAreErrors)
			{
				// How many unique previous errors we should keep track of
				const uint32 MaxPreviousErrorsToTrack = 4;
				static uint32 StaticPreviousErrorCount = 0;
				if (StaticPreviousErrorCount < MaxPreviousErrorsToTrack)
				{
					// Check to see if we've already reported this error. No point in blasting the server with
					// the same error over and over again in a single application session.
					bool bHasErrorAlreadyBeenReported = false;
 
					// Static: Array of previous unique error message CRCs
					static uint32 StaticPreviousErrorCRCs[MaxPreviousErrorsToTrack];
 
					// Compute CRC of error string. Note that along with the call stack, this includes the message
					// string passed to the macro, so only truly redundant errors will go unreported. Though it also
					// means you shouldn't pass loop counters to ensureMsgf(), otherwise failures may spam the server!
					const uint32 ErrorStrCRC = FCrc::StrCrc_DEPRECATED(ErrorMsg);
 
					for (uint32 CurErrorIndex = 0; CurErrorIndex < StaticPreviousErrorCount; ++CurErrorIndex)
					{
						if (StaticPreviousErrorCRCs[CurErrorIndex] == ErrorStrCRC)
						{
							// Found it! This is a redundant error message.
							bHasErrorAlreadyBeenReported = true;
							break;
						}
					}
 
					// Add the element to the list and bump the count
					StaticPreviousErrorCRCs[StaticPreviousErrorCount++] = ErrorStrCRC;
 
					if (!bHasErrorAlreadyBeenReported)
					{
#if STATS
						FString SubmitErrorReporterfMessage = FString::Printf(TEXT("SubmitErrorReport"));
						SCOPE_LOG_TIME_IN_SECONDS(*SubmitErrorReporterfMessage, nullptr)
#endif
 
						FCoreDelegates::OnHandleSystemEnsure.Broadcast();
 
						FPlatformMisc::SubmitErrorReport(ErrorMsg, EErrorReportMode::Balloon);
 
						bShouldSendNewReport = true;
					}
				}
			}

[Attachment Removed]

2

Steps to Reproduce

  1. Trigger more than 4 ensures.
  2. Observe that after 4 have been broadcast then OnHandleSystemEnsure will no longer be broadcast.
    [Attachment Removed]
3

Hi,

That for reporting the problem. I tried to find the reason the reporting is limited to 4 ensures and it’s going back to the early days of UE4. My guess would be that there was too many ensures and it was spamming the server at the time. It feels a bit weird to have such a limit. There is also an error in the logic as duplicates should not count toward the maximum. There original logic was reporting an unlimited number of ensures. It was the duplication check that covered the last 4 reports through a circular buffer.

I logged a bug so this can be reviewed by the team and requested that we add a way to lift the maximum through settings or cvar as a minimum. The case should eventually be visible on the public tracker at: https://issues.unrealengine.com/issue/UE\-400736

Regards,

Martin

[Attachment Removed]

4

Thanks, yes, initially I thought it was counting and limiting repeated ensures but it’s not, it’s just the first 4 ensures that hit this code path that use up the budget.

Glad to see that it is bugged, I think a fix here will improve visibility of issues and allow teams to prioritise them sooner.

[Attachment Removed]