How to correctly consume gamepad virtual accept input with Common UI

Development Programming & Scripting UI
, ,
1

Hello,

We are using CommonUI in our project and encountered some issue regarding the behaviour of the Virtual_Accept described here: CommonUI Input Technical Guide for Unreal Engine | Unreal Engine 5.6 Documentation | Epic Developer Community. From my understanding, if we have a CommonButton that has a binding on OnButtonBaseClicked, and is mapped to a TriggeringInputAction (DataTable) corresponding to the Virtual Accept key (“A” on Xbox controller), the CommonAnalogCursor will intercept the KeyDown event and generate LeftMouseButton “Down” event.

The issue is that, by the time the user releases the Virtual Accept key, we may close and pop the current activatable widget/panel, so another would activate below, and then the new panel may receive the Release/Key Up event, leading to another button triggering its OnButtonBaseClicked binding. This would be annoying because the user would unwillingly confirm two actions in a row.

I see in the FCommonAnalogCursor::HandleKeyUpEvent() that we evaluate as True this condition, if (!bIsVirtualAccept || ShouldVirtualAcceptSimulateMouseButton(InKeyEvent, IE_Released). My guess is that I should override in our own CommonAnalogCursor overridden class the ShouldVirtualAcceptSimulateMouseButton() method and return false for EInputEvent::IE_Released.

However, do you see any caveat to this approach, or am I missing something?

Best regards,

Baptiste

2

Hi,

What is the Click Method set to on these two buttons? By default we require Down and Up before processing a button press, so the first button would consume the full click from the analog cursor. That said, I suspect what is happening here is that you have input coming from two separate sources- the bound input action and the virtual click.

To take a step back, you typically shouldn’t need to bind an input action to a button if you plan to focus and “click” it with the analog cursor; triggering input action is more intended for a button which might not be focusable but can always be triggered with a specific button. You could disable CommonUI.ShouldVirtualAcceptSimulateMouseButton if you’d rather explicitly bind the button, but the better approach is probably to clear those bindings out if the intention is for the player to navigate to the button and press it. In the CommonAnalogCursor we have a bit of special handling where we try to see if there is a triggering input action bound to the button before we send the virtual click:

if (bIsVirtualAccept && ActionRouter.ProcessInput(InKeyEvent.GetKey(), InputEventType) == ERouteUIInputResult::Handled) { return true; }I imagine this is finding the bound action on the first button and triggering it directly (instead of “clicking it”) which leaves the release event to fall through to the next widget. Removing the binding altogether should let it send the virtual click like normal, which would then be processed based on the Click Method mentioned before (where we capture during mouse down and “press” the button on mouse up.

Best,

Cody

3

Hi,

Thanks for clarifying, it looks like my hunch was wrong. It does appear that input actions from the table can only respond to key down event (and I’m seeing the same inconsistency in Lyra between buttons using the virtual cursor and buttons using input actions) so the Down and Up input method won’t apply in that case. For what it’s worth, the eventual goal here is to flesh out the Enhanced Input support more, which would open the door for binding Enhanced Input Actions with much more control over the trigger conditions. However, that support is still quite limited and experimental, so if we can make the existing CommonUI input actions work then that’s a much safer approach for the time being.

I suppose that means the real mystery is why the key up event is triggering a bound button as well. Is the second button (under the modal) also being bound to the input action, or is it receiving focus and seeing the synthesized mouse up event? I’d expect the first case to be filtered since the event is IE_Released, and the second case to be filtered by the click method requiring down and up (and not reacting to a lone mouse up event). There’s still a bit of strangeness in the lone Mouse Up event being sent wherever the virtual cursor happens to be (presumably over the widget which was focused as part of the new widget being activated), but our goal would be to ensure it’s ignored by whichever widget it lands on. If that’s not what you’re seeing, maybe the Slate Debugger could help determine which events are being sent to the button to see if anything unexpected is happening.

Best,

Cody

4

That looks good, sorry yeah my snippet above was the change I made in main which has some other changes that required using the ActionRouter variable but what you have looks good for 5.5 and prior.

5

Hello Cody,

Thank you for your precise explanation. I didn’t change the Click Method on these buttons, that is why I expected the “Down and Up” strategy to be working for Virtual Accept flows. Concerning your suspicion, I checked and in the first modal, we have the “Confirm” button that isn’t focusable and two slot buttons that are focusable and can be navigated from one to the other with the D-Pad, but these don’t have any bound input action (Triggering Input Action). The “Confirm” button has a bound input action (Triggering Input Action) mapped to the Virtual Accept Key, with the mentioned issue. I attach a screenshot to help visualize the situation.

[Image Removed]If I remove the Triggering Input Action from this button, pressing A doesn’t do anything anymore, so I wonder if “input coming from two separate sources- the bound input action and the virtual click” is still a viable explanation?

With your answer, I think I better understand the difference between navigable/focusable buttons which should ideally only rely on virtual click (via analog cursor), versus buttons that aren’t included in the navigation grid and can only be accessed via other buttons than virtual accept on gamepad. However, with my configuration where I only have a bound and unfocusable button, I would expect the virtual accept flow to respect the default “Down and Up” strategy set. I had a look on the code and in the FActionRouterBindingCollection::ProcessNormalInput(ECommonInputMode ActiveInputMode, FKey Key, EInputEvent InputEvent) method, in the TryConsumeInput(const FKey& InKey, const UInputAction* InInputAction) lambda, we need to pass the following check: Binding->InputEvent == InputEvent. This seems wrong to me for two reasons:

  • When inspecting the FUIActionBinding::InputEvent field, I found that is is only built from a FBindUIActionArgs::KeyEvent field, which is never overriden in the case of the CommonButtons (see UCommonButtonBase::BindTriggeringInputActionToClick()), so it would stay as “EInputEvent::IE_Pressed”. So I don’t understand how a bound input action on a CommonButton could ever catch a Key Release event from a gamepad controller.
  • Even if I made a custom method to override/set a binding with the input event “EInputEvent::IE_Released” on a CommonButton, because of the duplicates check on FUIActionBinding::TryCreate(), we would need to duplicate in the Data Table the input action row for the same real input action, which seems odd.

So, with all of this being said, how is CommonUI correctly implementing the “Down and Up” strategy, regarding the Virtual Accept behaviour?

Thank you for your help and patience.

Best regards,

Baptiste

6

Hello Cody,

Thank you for your explanation. I understand the goal with Enhanced Input support. To be honest, we tried to activate the binding between both systems earlier we our project, but ended disabling it because of the many problems we encountered. We may reconsider our decision when it is less experimental.

I checked and the second button on the modal below is also bound to an input action mapped to the virtual accept key (Data Table), but without On Clicked delegate, like for the first modal. Indeed, this CTA is only cosmetic in gamepad. In this modal, we also have a list of navigable buttons that we manually bind in c++ on UCommonButtonBase::NativeOnClicked(), so the expected behaviour is that we navigate between them and commit our action using the “A” key (in Mouse and Keyboard, we just click the wanted button/line to select our wanted option, fowarding to the same NativeOnClicked() delegate). In any case, the CTA button never has focus, nor is intended to have it, I checked to be sure and its “bIsFocusable” is correctly set to “false”.

I placed a breakpoint on our override of the NativeOnClicked() callback to understand in which callstack the Key Release Event is called, and here is what I obtain:

[UnrealEditor-Brimstone-Win64-DebugGame.dll] UReviveOptionLine::NativeOnClicked() ReviveOptionLine.cpp:58 [UnrealEditor-CommonUI.dll] UCommonButtonBase::HandleButtonClicked() CommonButtonBase.cpp:1300 [UnrealEditor-CoreUObject.dll] UFunction::Invoke(UObject *, FFrame &, void *const) Class.cpp:7192 [Blueprint] CommonButtonBase: HandleButtonClicked Function /Script/CommonUI.CommonButtonBase:HandleButtonClicked [UnrealEditor-CoreUObject.dll] UObject::ProcessEvent(UFunction *, void *) ScriptCore.cpp:2173 [Inlined] [UnrealEditor-UMG.dll] TScriptDelegate<FNotThreadSafeNotCheckedDelegateMode>::ProcessDelegate(void *) ScriptDelegates.h:448 [UnrealEditor-UMG.dll] TMulticastScriptDelegate<FNotThreadSafeDelegateMode>::ProcessMulticastDelegate<UObject>(void *) ScriptDelegates.h:918 [Inlined] [UnrealEditor-UMG.dll] FOnButtonClickedEvent::Broadcast() Button.h:17 [UnrealEditor-UMG.dll] UButton::SlateHandleClicked() Button.cpp:259 [UnrealEditor-CommonUI.dll] UCommonButtonInternalBase::SlateHandleClickedOverride() CommonButtonBase.cpp:295 [Inlined] [UnrealEditor-CommonUI.dll] Invoke(FReply (UCommonButtonInternalBase::*)(), UCommonButtonInternalBase *&) Invoke.h:66 [Inlined] [UnrealEditor-CommonUI.dll] UE::Core::Private::Tuple::TTupleBase<TIntegerSequence<unsigned int> >::ApplyAfter(FReply (UCommonButtonInternalBase::*&)(), UCommonButtonInternalBase *&) Tuple.h:317 [UnrealEditor-CommonUI.dll] TBaseUObjectMethodDelegateInstance<0, UCommonButtonInternalBase, FReply __cdecl(void), FDefaultDelegateUserPolicy>::Execute() DelegateInstancesImpl.h:650 [Inlined] [UnrealEditor-Slate.dll] TDelegate<FReply __cdecl(void), FDefaultDelegateUserPolicy>::Execute() DelegateSignatureImpl.inl:613 [UnrealEditor-Slate.dll] SButton::ExecuteOnClick() SButton.cpp:467 [UnrealEditor-Slate.dll] SButton::OnMouseButtonUp(const FGeometry &, const FPointerEvent &) SButton.cpp:392 [UnrealEditor-CommonUI.dll] SCommonButton::OnMouseButtonUp(const FGeometry &, const FPointerEvent &) CommonButtonTypes.cpp:61 [UnrealEditor-Slate.dll] FSlateApplication::RoutePointerUpEvent’::8'::<lambda_2>::operator()(const FArrangedWidget &,const FPointerEvent &) SlateApplication.cpp:5346 [UnrealEditor-Slate.dll] FEventRouter::Route<FReply,FEventRouter::FToLeafmostPolicy,FPointerEvent,FSlateApplication::RoutePointerUpEvent’::8'::<lambda_2> >(FSlateApplication *,FToLeafmostPolicy,FPointerEvent,const <lambda_2> &,ESlateDebuggingInputEvent) SlateApplication.cpp:456 [UnrealEditor-Slate.dll] FSlateApplication::RoutePointerUpEvent(const FWidgetPath &, const FPointerEvent &) SlateApplication.cpp:5332 [UnrealEditor-Slate.dll] FSlateApplication::ProcessMouseButtonUpEvent(const FPointerEvent &) SlateApplication.cpp:5917 [UnrealEditor-Slate.dll] FAnalogCursor::HandleKeyUpEvent(FSlateApplication &, const FKeyEvent &) AnalogCursor.cpp:118 [UnrealEditor-CommonUI.dll] FCommonAnalogCursor::HandleKeyUpEvent(FSlateApplication &, const FKeyEvent &) CommonAnalogCursor.cpp:298 [Inlined] [UnrealEditor-Slate.dll] UE::Core::Private::Function::TFunctionRefBase<UE::Core::Private::Function::FFunctionRefStoragePolicy, bool __cdecl(IInputProcessor &)>::operator()(IInputProcessor &) Function.h:470 [UnrealEditor-Slate.dll] FSlateApplication::InputPreProcessorsHelper::PreProcessInput(ESlateDebuggingInputEvent, TFunctionRef<bool __cdecl(IInputProcessor &)>) SlateApplication.cpp:7596 [Inlined] [UnrealEditor-Slate.dll] FSlateApplication::InputPreProcessorsHelper::HandleKeyUpEvent(FSlateApplication &, const FKeyEvent &) SlateApplication.cpp:7420 [UnrealEditor-Slate.dll] FSlateApplication::ProcessKeyUpEvent(const FKeyEvent &) SlateApplication.cpp:4874 [UnrealEditor-Slate.dll] FSlateApplication::OnControllerButtonReleased(FName, FPlatformUserId, FInputDeviceId, bool) SlateApplication.cpp:6402 [UnrealEditor-XInputDevice.dll] XInputInterface::SendControllerEvents() XInputInterface.cpp:256 [UnrealEditor-ApplicationCore.dll] FWindowsApplication::PollGameDeviceState(const float) WindowsApplication.cpp:2860 [UnrealEditor-Slate.dll] FSlateApplication::PollGameDeviceState() SlateApplication.cpp:1443 [UnrealEditor-Win64-DebugGame.exe] FEngineLoop::Tick() LaunchEngineLoop.cpp:5861 [Inlined] [UnrealEditor-Win64-DebugGame.exe] EngineTick() Launch.cpp:69 [UnrealEditor-Win64-DebugGame.exe] GuardedMain(const wchar_t *) Launch.cpp:190 [UnrealEditor-Win64-DebugGame.exe] LaunchWindowsStartup(HINSTANCE__ *, HINSTANCE__ *, char *, int, const wchar_t *) LaunchWindows.cpp:266 [UnrealEditor-Win64-DebugGame.exe] WinMain(HINSTANCE__ *, HINSTANCE__ *, char *, int) LaunchWindows.cpp:317Basically, we call FAnalogCursor::HandleKeyUpEvent(SlateApp, InKeyEvent) for the Key up event in FCommonAnalogCursor::HandleKeyUpEvent(FSlateApplication& SlateApp, const FKeyEvent& InKeyEvent) (see [Content removed] for the custom modifications we made in the snippet):

`bool FCommonAnalogCursor::HandleKeyUpEvent(FSlateApplication& SlateApp, const FKeyEvent& InKeyEvent)
{
if (IsRelevantInput(InKeyEvent))
{
#if !UE_BUILD_SHIPPING
const FKey& PressedKey = InKeyEvent.GetKey();
if (PressedKey == EKeys::Gamepad_LeftShoulder) { ShoulderButtonStatus ^= EShoulderButtonFlags::LeftShoulder; }
if (PressedKey == EKeys::Gamepad_RightShoulder) { ShoulderButtonStatus ^= EShoulderButtonFlags::RightShoulder; }
if (PressedKey == EKeys::Gamepad_LeftTrigger) { ShoulderButtonStatus ^= EShoulderButtonFlags::LeftTrigger; }
if (PressedKey == EKeys::Gamepad_RightTrigger) { ShoulderButtonStatus ^= EShoulderButtonFlags::RightTrigger; }
#endif

// We support binding actions to the virtual accept key, so it’s a special flower that gets processed right now
const bool bIsVirtualAccept = InKeyEvent.GetKey() == EKeys::Virtual_Accept;
if (bIsVirtualAccept && ActionRouter.ProcessInput(InKeyEvent.GetKey(), IE_Released) == ERouteUIInputResult::Handled)
{
//TA>
bHandledVirtualAcceptDown = false;
//<TA
return true;
}
else if (!bIsVirtualAccept || ShouldVirtualAcceptSimulateMouseButton(InKeyEvent, IE_Released))
{
//TA>
if (bIsVirtualAccept && bHandledVirtualAcceptDown)
{
UCommonInputSubsystem& InputSubsytem = ActionRouter.GetInputSubsystem();
InputSubsytem.SetIsGamepadSimulatedClick(bIsVirtualAccept);
bool bReturnValue = FAnalogCursor::HandleKeyDownEvent(SlateApp, InKeyEvent);
InputSubsytem.SetIsGamepadSimulatedClick(false);
bHandledVirtualAcceptDown = false;
}
//<TA
return FAnalogCursor::HandleKeyUpEvent(SlateApp, InKeyEvent);
}
}
return false;
}`So, if I understand what is happening correctly, we would expect/want this lone event to be absorbed and somehow handled in this line:

if (bIsVirtualAccept && ActionRouter.ProcessInput(InKeyEvent.GetKey(), IE_Released) == ERouteUIInputResult::Handled)

but we don’t get a ERouteUIInputResult::Handled as a result, thus falling back to the line

else if (!bIsVirtualAccept || ShouldVirtualAcceptSimulateMouseButton(InKeyEvent, IE_Released))

from which stems my original post.

7

From there, since default implementation of FCommonAnalogCursor::ShouldVirtualAcceptSimulateMouseButton() returns true, we finally call FAnalogCursor::HandleKeyUpEvent(SlateApp, InKeyEvent) and handle this event as a mouse button up event (SlateApp.ProcessMouseButtonUpEvent(MouseEvent) call). The one thing that looked odd to me was the call to SButton::ExecuteOnClick() that goes through. Even if we are in the synthetic cursor flow here, I would expect this requirement

const bool bMeetsPressedRequirements = (!bMustBePressed || (bIsPressed && bMustBePressed));

to fail, but for some reason, bIsPressed is raised to true when I release the “A” key. I put another breakpoint to check what is happening, and this comes from this snippet in FCommonAnalogCursor::HandleKeyUpEvent(FSlateApplication& SlateApp, const FKeyEvent& InKeyEvent):

else if (!bIsVirtualAccept || ShouldVirtualAcceptSimulateMouseButton(InKeyEvent, IE_Released)) { //TA> if (bIsVirtualAccept && bHandledVirtualAcceptDown) { UCommonInputSubsystem& InputSubsytem = ActionRouter.GetInputSubsystem(); InputSubsytem.SetIsGamepadSimulatedClick(bIsVirtualAccept); bool bReturnValue = FAnalogCursor::HandleKeyDownEvent(SlateApp, InKeyEvent); InputSubsytem.SetIsGamepadSimulatedClick(false); bHandledVirtualAcceptDown = false; } //<TA return FAnalogCursor::HandleKeyUpEvent(SlateApp, InKeyEvent); }More precisely, this comes from the call of bool bReturnValue = FAnalogCursor::HandleKeyDownEvent(SlateApp, InKeyEvent);

So I guess, this custom snippet may be the culprit for the unintended behaviour. Especially, since we are navigating between buttons in this second modal, maybe it is logical that this line if (bIsVirtualAccept && ActionRouter.ProcessInput(InKeyEvent.GetKey(), IE_Released) == ERouteUIInputResult::Handled) fails and we really want to handle the situation with the synthetic cursor in the block inside this condition:

else if (!bIsVirtualAccept || ShouldVirtualAcceptSimulateMouseButton(InKeyEvent, IE_Released))

What are you thoughts on the matter?

Best regards,

Baptiste

8

Hey there,

I believe I’ve found a solution that will solve this problem and your original problem. I’ll be submitting this change to the engine but here’s the details if you’d like to make the change locally. You should first revert the bHandledVirtualAcceptDown change you made.

In FCommonAnalogCursor add these fields:

`void OnVirtualAcceptHoldCanceled();

TOptional ActiveKeyUpEvent;`And add this include to CommonAnalogCursor.h:

#include "Input/Events.h"Implement OnVirtualAcceptHoldCanceled():

void FCommonAnalogCursor::OnVirtualAcceptHoldCanceled() { if (ActiveKeyUpEvent.IsSet() && ActiveKeyUpEvent->GetKey() == EKeys::Virtual_Accept && ShouldVirtualAcceptSimulateMouseButton(ActiveKeyUpEvent.GetValue(), IE_Pressed)) { UCommonInputSubsystem& InputSubsystem = ActionRouter.GetInputSubsystem(); InputSubsystem.SetIsGamepadSimulatedClick(true); FAnalogCursor::HandleKeyDownEvent(FSlateApplication::Get(), ActiveKeyUpEvent.GetValue()); InputSubsystem.SetIsGamepadSimulatedClick(false); } }Add this line to FCommonAnalogCursor::HandleKeyUpEvent, before const bool bIsVirtualAccept:

`TGuardValue<TOptional> KeyUpEventGuard(ActiveKeyUpEvent, TOptional(InKeyEvent));

// We support binding actions to the virtual accept key, so it’s a special flower that gets processed right now
const bool bIsVirtualAccept = InKeyEvent.GetKey() == EKeys::Virtual_Accept;`In UCommonUIActionRouterBase::ProcessInput, change the if (ProcessHoldResult == EProcessHoldActionResult::GeneratePress) block to this:

if (ProcessHoldResult == EProcessHoldActionResult::GeneratePress) { // A hold action was in progress but quickly aborted, so we want to generate a press action now for any normal bindings that are interested if (!ProcessNormalInputFunc(ActionRouter, IE_Pressed)) { if (Key == EKeys::Virtual_Accept && ActionRouter.AnalogCursor.IsValid()) { // If Virtual_Accept is bound to a Hold action, the Pressed event for it will not trigger a simulated click // but when a hold action is canceled fast enough, we want to trigger any "normal" actions bound to the same key // so we give the analog cursor a chance to trigger the simulated click here // This allows users to bind Hold actions to Virtual_Accept while still triggering focused button widgets with quick virtual_accept presses ActionRouter.AnalogCursor->OnVirtualAcceptHoldCanceled(); } } }And then you should be good to go. Your buttons will still trigger when there’s a cancelled Hold action bound to Virtual_Accept and you won’t get extra simulated Click events otherwise.

9

Hi Dylan,

Thank you for your precise answer. I’m off vacation tonight, but I’ll be able to test your fix in two weeks.

Best regards and thank you again!

Baptiste

10

Hi Dylan,

I had the chance to test your fix. It seems to work well and fix both of our issues, thank you very much! The snippet in our side looks like this, without mentions of “ActionRouter” variable, because we are already in the UCommonUIActionRouterBase scope. It works just fine like this, but please let me know if I missed something!

if (ProcessHoldResult == EProcessHoldActionResult::GeneratePress) { // A hold action was in progress but quickly aborted, so we want to generate a press action now for any normal bindings that are interested if (!ProcessNormalInputFunc(IE_Pressed)) { if (Key == EKeys::Virtual_Accept && AnalogCursor.IsValid()) { // If Virtual_Accept is bound to a Hold action, the Pressed event for it will not trigger a simulated click // but when a hold action is canceled fast enough, we want to trigger any "normal" actions bound to the same key // so we give the analog cursor a chance to trigger the simulated click here // This allows users to bind Hold actions to Virtual_Accept while still triggering focused button widgets with quick virtual_accept presses AnalogCursor->OnVirtualAcceptHoldCanceled(); } } }Best regards,

Baptiste