Enhanced Input crash in RebuildControlMappings

We have been getting rare crashes for years in the EnhancedInput Plugin that have only really happened to our players that we have ONE instance that we have gotten while visual studio was attached (just yesterday).

Our best assessment is that one of the IMCs had recently been removed, and had been destroyed by garbage collection roughly at the same time as reordermappings was reassessing, before it had been fully removed.

We’re using a child of EnhancedInput with Key Rebindings actually, and I can see 5 IMCs in the Applied IMC list there, and in OrderedInputContexts, I could see 6 entries, with one in the middle having bogus data.

We have actually have gotten 5 or so different callstacks that all seem to be based on different ways accessing bogus data can crash.

Right now, we’re considering bandaids for avoiding those various dangling pointers. We do not know if we can avoid those dangling pointers at this time… And an engine fix, we’re not certain if we could get improvements without doing another full engine upgrade.

We figure that while the pointer is bad, and we are likely to crash, because the theory is that this on removal, we can get away with a graceful avoidance for a subset of detectable scenarios. This is my present proposed bandaid.

	for (const TPair<TObjectPtr<const UInputMappingContext>, int32>& ContextPair : OrderedInputContexts)
	{
		// Don't apply context specific keys immediately, allowing multiple mappings to the same key within the same context if required.
		TArray<FKey> ContextAppliedKeys;
		const UInputMappingContext* MappingContext = ContextPair.Key;

		// START BANDAID
		if (!IsValid(MappingContext))
		{
			continue;
		}

		if (!MappingContext->IsValidLowLevelFast()) //jlisco - suspected way to detect a dangling pointer... Nice in that it checks for some memory offset from null.
		{
			checkf(MappingContext->IsValidLowLevelFast(), TEXT("Rebuild Control Mappings encountered IMC %s that is failed IsValidLowLevelFast. Priority was %d with %d total IMCs in OrderedInputContexts. Had flags %d."),
				*GetNameSafe(MappingContext), ContextPair.Value, OrderedInputContexts.Num(), (int32)(MappingContext->GetFlags()));
			continue;
		}

		if (!MappingContext->IsValidLowLevel()) //jlisco - suspected way to detect a dangling pointer... Different from LowLevelFast in that it will check if the object array is actually pointing to this directly.
		{
			checkf(MappingContext->IsValidLowLevel(), TEXT("Rebuild Control Mappings encountered IMC %s that is failed IsValidLowLevel. Priority was %d with %d total IMCs in OrderedInputContexts. Had flags %d."),
				*GetNameSafe(MappingContext), ContextPair.Value, OrderedInputContexts.Num(), (int32)(MappingContext->GetFlags()));
			continue;
		}

		if (MappingContext->HasAnyFlags(RF_BeginDestroyed | RF_FinishDestroyed)) //jlisco - Known incomplete catch of a subscenario of crashing. Seems like a dangling pointer in the map somehow (other elements in the list are valid)
		{
			checkf(!MappingContext->HasAnyFlags(RF_FinishDestroyed), TEXT("Rebuild Control Mappings encountered IMC %s with Finish Destroyed Flag set. Priority was %d with %d total IMCs in OrderedInputContexts"),
				*GetNameSafe(MappingContext), ContextPair.Value, OrderedInputContexts.Num());
			checkf(!MappingContext->HasAnyFlags(RF_BeginDestroyed), TEXT("Rebuild Control Mappings encountered IMC %s with Begin Destroyed Flag set. Priority was %d with %d total IMCs in OrderedInputContexts"),
				*GetNameSafe(MappingContext), ContextPair.Value, OrderedInputContexts.Num());
			continue;
		}

		if (MappingContext->GetMappings().Num() > 10000) //jlisco - trying to catch more information about a client crash report... but probably this was another type of dangling pointer...
		{
			checkf(MappingContext->GetMappings().Num() <= 10000, TEXT("Rebuild Control Mappings encountered IMC %s with %d internal mappings (restricting max count at this time). Priority was %d with %d total IMCs in OrderedInputContexts"), 
				*GetNameSafe(MappingContext), MappingContext->GetMappings().Num(), ContextPair.Value, OrderedInputContexts.Num());
			continue;
		}

Are there any other recommendations here?

Steps to Reproduce
Rare (probably garbage collect related) event when removing an input mapping context.

Heya, do you have any of the callstacks or logs which you could share? We can convert to a private question if necessary. I haven’t seen this crash reported elsewhere, but if you have a callstack then I could search our internal crash reporter database for it.

This would be an odd case for a GC issue, because the OrderedInputContexts is populated by a UPROPERTY and it all appears to be keeping the references with the correct UPROPERTY markup that I can see

Thanks for this report!

Dredging up the old thread on this… I recall that we found that something was unsafe with running the tick externally, so garbage collect could conflict or something…

Let me try attaching some images…

You can see some of the code myself and another person tried to inject to gracefully handling these situations, but I think that’s fine.

[Image Removed][Image Removed]

Oh here’s an example of a customer reported callstack that triggered the following assert:

Fatal error: [File:D:\shared\Palia\Palia-release\Engine\Source\Runtime\Core\Private\Containers\ContainerHelpers.cpp] [Line: 8]

Trying to resize TArray to an invalid size of 2547790696

Callstack:

Crashed in non-app KERNELBASE +0x000c7f7a RaiseException

+0x0174071b ReportAssert (WindowsPlatformCrashContext.cpp:1874)

+0x0174071b FWindowsErrorOutputDevice::Serialize (WindowsErrorOutputDevice.cpp:82)

+0x015c82be FOutputDevice::LogfImpl (OutputDevice.cpp:81)

+0x014832ec UE::Logging::Private::BasicFatalLog (StructuredLog.cpp:1104)

+0x0129b229 UE::Core::Private::OnInvalidArrayNum (ContainerHelpers.cpp:8)

+0x0a53cd32 TArray<T>::Reserve (Array.h:2646)

+0x0a53cd32 IEnhancedInputSubsystemInterface::ReorderMappings (EnhancedInputSubsystemInterface.cpp:682)

+0x0a535142 IEnhancedInputSubsystemInterface::RebuildControlMappings (EnhancedInputSubsystemInterface.cpp:840)

+0x0a51e77c FEnhancedInputModule::Tick::__l2::<T>::operator() (EnhancedInputModule.cpp:318)

+0x0a51e77c Invoke (Invoke.h:47)

+0x0a51e77c UE::Core::Private::Function::TFunctionRefCaller<T>::Call (Function.h:315)

+0x0a526eb9 UE::Core::Private::Function::TFunctionRefBase<T>::operator() (Function.h:470)

+0x0a526eb9 UEnhancedInputLibrary::ForEachSubsystem (EnhancedInputLibrary.cpp:32)

+0x0a4f9de1 FEnhancedInputModule::Tick (EnhancedInputModule.cpp:313)

+0x083b1dfa FTickableGameObject::TickObjects (Tickable.cpp:196)

+0x0762bba3 UGameEngine::Tick (GameEngine.cpp:1852)

+0x087db624 FEngineLoop::Tick (LaunchEngineLoop.cpp:5871)

+0x087ec0cb EngineTick (Launch.cpp:69)

+0x087ec0cb GuardedMain (Launch.cpp:190)

+0x087ec199 GuardedMainWrapper (LaunchWindows.cpp:123)

+0x087eeec1 LaunchWindowsStartup (LaunchWindows.cpp:277)

+0x087fbdf3 WinMain (LaunchWindows.cpp:317)

Ok, if you are correctly using the UPROPERTY parts then you should be alright.

For what its worth, I have added the IsValid check into the engine, which you can see on GitHub here: https://github.com/EpicGames/UnrealEngine/commit/a465fb7f029a1b22e3dcdfbf35167608d0c3c66d

Thanks for the bug report! I made it an ensure, so hopefully we will be able to get some CrashReporter data on it in the future to discover the root cause. I’m going to close this ticket for now

Hey John,

Thanks for the additional info! I will keep an eye on crash reporter to see if there are others with this issue.

In the meantime, I would still be inclined to say that it is something to do with how your IMC object is being created at runtime. Can you post how you are using NewObject? We can convert to a confidential question if you would like

Thanks,

Ben

Thanks for the additional info here, unfortunately there isn’t much I can do here for you. I don’t see any instances of this specific callstack within our crash reporter system for more data, so the issue would appear to be just in your project and not very wide spread.

(Whoops posted as new answers rather than threaded)

That’s interesting! I don’t see any issue with your proposed “IsValid” check there, but I’m not sure how it would be getting garbage collected, and I haven’t seen this happen on any of our project so far. It looks to me that everything is correctly marked as a UPROPERTY where it needs to be, so nothing is jumping out to me from within the Enhanced Input System that would be causing GC issues.

I noticed that check you have there for “if mappings > 10000, skip)”… are you creating your UInputMappingContext’s dynamically via C++? Or do you truly have an asset that has 10k mappings in it!? :slight_smile:

I ask because I’m wondering if maybe that dynamic generation bit could be root the problem.

I will submit your suggested IsValid fix in the meantime.

Thanks,

Ben

We have one Input Mapping that can be formed dynamically, that can have a max of 4 input actions inside of it. This was incorrect, apparently we choose between several authored input mapping contexts dynamically.

The rest are all explicitly authored with well under 100 each.

The assumption is that if you see an inordinate number of mappings, that input context was probably on the way out, and would be intended to be removed, but is unfortunately corrupt for the one frame.

I can look more closely at how that one dynamically formed input context is made.

Ok, looking more closely… We have a proximity based interactor component, and when an interactable component is nearby, it asks them what input mapping context it wants to use, which apparently in all use cases is actually authored input mapping contexts in configs.

It’ll add/remove based on what is nearby (or is no longer nearby).

It stores the Input Contexts in a local Transient UProperty array at the same time as it registers it with the subsystem.

When it changes later, it clears the array and rebuilds it based on the same request functions.

Oh as a heads up, the one IsValid proved itself insufficient to resolve the crash. (We’ve been trying to resolve this piece by piece for months)

The case where I got it locally led me to add the IsValidLowLevel and IsValidLowLevelFast, as well as double checking object flags.

It is not actually dynamically created. I misrepresented that earlier, but clarified in a previous threaded reply…

Looked like:

Ok, looking more closely… We have a proximity based interactor component, and when an interactable component is nearby, it asks them what input mapping context it wants to use, which apparently in all use cases is actually authored input mapping contexts in configs.

It’ll add/remove based on what is nearby (or is no longer nearby).

It stores the Input Contexts in a local Transient UProperty array at the same time as it registers it with the subsystem.

When it changes later, it clears the array and rebuilds it based on the same request functions.

Separately from that, we got a crash instance even when using IsValidLowLevelFast after IsValid passed.

  • IsObjectHandleNull (ObjectHandle.h:101)
  • bool (ObjectPtr.h:187)
  • ObjectPtr_Private::IsObjectPtrNull (ObjectPtr.h:397)
  • TObjectPtr<T>::operator== (ObjectPtr.h:620)
  • UObjectBase::IsValidLowLevelFast (UObjectBase.cpp:360)
  • UObjectBase::IsValidLowLevelFast (UObjectBase.cpp:373)
  • IEnhancedInputSubsystemInterface::RebuildControlMappings (EnhancedInputSubsystemInterface.cpp:832)
  • FEnhancedInputModule::Tick::__l2::<T>::operator() (EnhancedInputModule.cpp:318)
  • Invoke (Invoke.h:47)
  • UE::Core::Private::Function::TFunctionRefCaller<T>::Call (Function.h:315)
  • UE::Core::Private::Function::TFunctionRefBase<T>::operator() (Function.h:470)
  • UEnhancedInputLibrary::ForEachSubsystem (EnhancedInputLibrary.cpp:32)
  • FEnhancedInputModule::Tick (EnhancedInputModule.cpp:313)
  • FTickableGameObject::TickObjects (Tickable.cpp:196)
  • UGameEngine::Tick (GameEngine.cpp:1852)
  • FEngineLoop::Tick (LaunchEngineLoop.cpp:5871)
  • EngineTick (Launch.cpp:69)
  • GuardedMain (Launch.cpp:190)
  • GuardedMainWrapper (LaunchWindows.cpp:123)
  • LaunchWindowsStartup (LaunchWindows.cpp:277)
  • WinMain (LaunchWindows.cpp:317)
  • invoke_main (exe_common.inl:102)
  • Show 3 more frames