Bubbling NativeOnMouseButtonDown from custom widget to PlayerController

Nikitaenterprise · October 5, 2021, 7:43am

Hi to all of you

If you create a custom widget for drawing selection rectangle and try to process mouse input (LMB for example) with NativeOnMouseButtonDown event the system will not pass information about mouse being pressed to player controller because of NativeOnMouseButtonDown returns FReply::Handled() (it’s needed for processing NativeOnMouseButtonUp event).

If NativeOnMouseButtonDown returns FReply::Unhandled() then PlayerController will get information about LMB were pressed but NativeOnMouseButtonUp won’t fire

Steps to reproduce

Create class UMyUserWidget : public UUserWidget.
Fill MyUserWidget.h and MyUserWidget.cpp with provided code.
In UE editor add new blueprint which parent is UMyUserWidget calling it NewBlueprint.
Add new basic WidgetBlueprint calling it NewWidgetBlueprint.
In NewWidgetBlueprint delete CanvasPanel and add NewBlueprint as a child. Set visibility of NewWidgetBlueprint to Not Hit-Testable (Self Only) on right side of UMG Designer
In GameMode blueprint add nodes as in Fig.1.
Start simulation and press and hold LMB after 1 second you will be able to draw blue rectangle and as you could see the time from PlayerController->GetInputKeyTimeDown(EKeys::LeftMouseButton) will be zero. If you press RMB then you will see that PlayerController->GetInputKeyTimeDown(EKeys::RightMouseButton) will raise from zero to some number, thus PlayerController knowing that RMB is pressed and LMB isn’t. If you double click with LMB then PlayerController->GetInputKeyTimeDown(EKeys::LeftMouseButton) will show time sinse first click and won’t stop.

What should I do to be able to get time of key pressed in PlayerController? In other words what should I do to pass handled event from widget to PlayerController (quick remind, that NativeOnMouseButtonUp should necessarily fire)?

MyUserWidget.h

     #pragma once
     
     #include "CoreMinimal.h"
     #include "Blueprint/UserWidget.h"
     #include "MyUserWidget.generated.h"
     
     class ARTSPlayerController;
     
     UCLASS()
     class RTSPROJECT_API UMyUserWidget : public UUserWidget
     {
         GENERATED_BODY()
     
     public:
     
         UPROPERTY(BlueprintReadOnly)
         ARTSPlayerController* PlayerController = nullptr;
     
     private:
     
         FVector2D StartClick;
         FVector2D HoldingLocation;
         bool bIsDrawingSelectionRectangle = false;
         bool bIsLMBPressed = false;
         float StartClickTime = 0;
         float HoldTime = 1;
     
         virtual void NativeConstruct() override;
         virtual void NativeTick(const FGeometry& MyGeometry, float InDeltaTime) override;
         virtual int32 NativePaint(const FPaintArgs& Args,
             const FGeometry& AllottedGeometry,
             const FSlateRect& MyCullingRect,
             FSlateWindowElementList& OutDrawElements,
             int32 LayerId,
             const FWidgetStyle& InWidgetStyle,
             bool bParentEnabled) const override;
     
         virtual FReply NativeOnMouseButtonDown(const FGeometry& InGeometry, const FPointerEvent& InMouseEvent) override;
         virtual FReply NativeOnMouseMove(const FGeometry& InGeometry, const FPointerEvent& InMouseEvent) override;
         virtual FReply NativeOnMouseButtonUp(const FGeometry& InGeometry, const FPointerEvent& InMouseEvent) override;
         
     };

MyUserWidget.cpp

 #include "UI/MyUserWidget.h"
 
 #include "Core/RTSPlayerController.h"
 
 void UMyUserWidget::NativeConstruct()
 {
     Super::NativeConstruct();
     ARTSPlayerController* TestPlayerController = Cast<ARTSPlayerController>(GetOwningPlayer());
     if (TestPlayerController)
     {
         PlayerController = TestPlayerController;
     }
     // For NativeOnKeyDown event to work properly the flag IsFocusable should be true in Widget->Designer->Interaction
     bIsFocusable = true;
 
     Visibility = ESlateVisibility::Visible;
 }
 
 void UMyUserWidget::NativeTick(const FGeometry& MovieSceneBlends, float InDeltaTime)
 {
     GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, FString::Printf(TEXT("PlayerController->GetInputKeyTimeDown(EKeys::LeftMouseButton) is %s"), *FString::SanitizeFloat(PlayerController->GetInputKeyTimeDown(EKeys::LeftMouseButton))));
 GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, FString::Printf(TEXT("PlayerController->GetInputKeyTimeDown(EKeys::RightMouseButton) is %s"), *FString::SanitizeFloat(PlayerController->GetInputKeyTimeDown(EKeys::RightMouseButton))));
 GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, FString::Printf(TEXT("bIsLMBPressed = %s"), bIsLMBPressed ? *FString("true") : *FString("false")));
 GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, FString::Printf(TEXT("bIsDrawingSelectionRectangle = %s"), bIsDrawingSelectionRectangle ? *FString("true") : *FString("false")));
 
 
     if (bIsLMBPressed)
     {
         const float CurrentTime = GetWorld()->GetTimeSeconds() - StartClickTime;
         if (CurrentTime < HoldTime)
         {
             GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, TEXT("Just click"));
         }
         else
         {
             GEngine->AddOnScreenDebugMessage(-1, 0.01, FColor::White, TEXT("Just drawing"));
             bIsDrawingSelectionRectangle = true;
         }
     }
     Super::NativeTick(MovieSceneBlends, InDeltaTime);
 }
 
 int32 UMyUserWidget::NativePaint(const FPaintArgs& MovieSceneBlends, const FGeometry& AllottedGeometry,
     const FSlateRect& MyCullingRect, FSlateWindowElementList& OutDrawElements, int32 LayerId,
     const FWidgetStyle& InWidgetStyle, bool bParentEnabled) const
 {
     LayerId = Super::NativePaint(MovieSceneBlends, AllottedGeometry, MyCullingRect, OutDrawElements, LayerId, InWidgetStyle, bParentEnabled);
     FPaintContext Context(AllottedGeometry, MyCullingRect, OutDrawElements, LayerId, InWidgetStyle, bParentEnabled);
     Context.MaxLayer++;
 
     if (bIsDrawingSelectionRectangle)
     {
         TArray<FVector2D> Points;
         Points.Add(FVector2D(StartClick.X, StartClick.Y));
         Points.Add(FVector2D(HoldingLocation.X, StartClick.Y));
         Points.Add(FVector2D(HoldingLocation.X, HoldingLocation.Y));
         Points.Add(FVector2D(StartClick.X, HoldingLocation.Y));
         Points.Add(FVector2D(StartClick.X, StartClick.Y));
 
         constexpr float Thickness = 1;
 
         FSlateDrawElement::MakeLines(
             Context.OutDrawElements,
             Context.MaxLayer,
             Context.AllottedGeometry.ToPaintGeometry(),
             Points,
             ESlateDrawEffect::None,
             FLinearColor::Blue,
             true,
             Thickness);
     }
 
     return FMath::Max(LayerId, Context.MaxLayer);
 }
 
 FReply UMyUserWidget::NativeOnMouseButtonDown(const FGeometry& MovieSceneBlends, const FPointerEvent& InMouseEvent)
 {
     GEngine->AddOnScreenDebugMessage(-1, 1, FColor::Blue, TEXT("UMyUserWidget::NativeOnMouseButtonDown"));
     auto Reply = Super::NativeOnMouseButtonDown(MovieSceneBlends, InMouseEvent);
     if (Reply.IsEventHandled())
     {
         return Reply;
     }
     if (InMouseEvent.GetPressedButtons().Num() == 1 && InMouseEvent.GetPressedButtons().Contains(EKeys::LeftMouseButton))
     {
         StartClickTime = GetWorld()->GetTimeSeconds();
         StartClick = MovieSceneBlends.AbsoluteToLocal(InMouseEvent.GetScreenSpacePosition());
         HoldingLocation = StartClick;
         bIsLMBPressed = true;
         return FReply::Handled();
     }
     return FReply::Unhandled();
 }
 
 FReply UMyUserWidget::NativeOnMouseMove(const FGeometry& MovieSceneBlends, const FPointerEvent& InMouseEvent)
 {
     auto Reply = Super::NativeOnMouseMove(MovieSceneBlends, InMouseEvent);
     if (Reply.IsEventHandled())
     {
         return Reply;
     }
     if (bIsDrawingSelectionRectangle)
     {
         HoldingLocation = MovieSceneBlends.AbsoluteToLocal(InMouseEvent.GetScreenSpacePosition());
     }
     return FReply::Unhandled();
 }
 
 FReply UMyUserWidget::NativeOnMouseButtonUp(const FGeometry& MovieSceneBlends, const FPointerEvent& InMouseEvent)
 {
     GEngine->AddOnScreenDebugMessage(-1, 1, FColor::Blue, TEXT("UMyUserWidget::NativeOnMouseButtonUp"));
     auto Reply = Super::NativeOnMouseButtonUp(MovieSceneBlends, InMouseEvent);
     if (Reply.IsEventHandled())
     {
         return Reply;
     }
     bIsDrawingSelectionRectangle = false;
     bIsLMBPressed = false;
     StartClickTime = 0;
     return FReply::Handled();
 }

Fig.1

and-rad · October 5, 2021, 9:32am

If you really only care about passing the time of the button press to the controller, how about just calling the player controller from the widget and passing the data? You can do that with UUserWidget::GetOwningPlayer() at a point that’s convenient.

Nikitaenterprise · October 5, 2021, 10:21am

No, I care about passing click event to PlayerController after handling that click in widget. The clock is only for representation purpose.

and-rad · October 5, 2021, 1:33pm

How static is the association between the pyhsical key/button and the event? Do you ever plan on allowing a key other than the left mouse button to perform that action? If not, you could create a function like AMyPlayerController::OnLeftMouseButtonClicked() and call that one from inside the widget. If it should be more generic, you can make it AMyPlayerController::OnKeyPressed(const FKey& KeyThatWasPressed) or something like that.

Nikitaenterprise · October 5, 2021, 3:01pm

Yes, that’s one of the possible solutions, and it’s ok until there are a solution that does something with bubbling event. The question not in ''how to tell PlayerController about LMB pressed?" but rather about how to handle NativeOnMouseButtonDown and NativeOnMouseButtonUp simultaneously and send that event to PlayerController

and-rad · October 6, 2021, 7:44am

Unless I’m grossly misunderstanding, that would work the same way. The signature of those UMG functions is

FReply NativeOnMouseButtonDown(const FGeometry& , const FPointerEvent&)
FReply NativeOnMouseButtonUp(const FGeometry& , const FPointerEvent&)

So you just implement a function on you player controller that expects these types as arguments.

void AMyPlayerController::OnInputEventReceived(const FPointerEvent& Event)

Whenever you want to pass the event to the player controller from A UUserWidget-derived class, you do

if (AMyPlayerController* Ctrl = Cast<AMyPlayerController>(GetOwningPlayer())
{
    Ctrl->OnInputEventReceived(InMouseEvent);
}

Nikitaenterprise · October 6, 2021, 8:38am

Yeah, I totally understand your proposition and agree that it will work perfectly, but…
I’ll try to explain the problem in a bit different way. The PlayerController by itself has an InputComponent (because APlayerController is inherited from AController which is inherited from AActor which has Input component as a public member) and that component can

handles input for this actor, if input is enabled

If you override SetupInputComponent() in your PlayerController class and try to bind clicks of LMB to some functions like this:

InputComponent->BindAction(TEXT("LMB"), IE_Pressed, this, &ARTSPlayerController::LMBPressed);
InputComponent->BindAction(TEXT("LMB"), IE_Released, this, &ARTSPlayerController::LMBReleased);

and, for sake of visualization, implement those functions like this:

void ARTSPlayerController::LMBPressed()
{
	GEngine->AddOnScreenDebugMessage(-1, 1, FColor::Cyan, TEXT("LMBPressed"));
}

void ARTSPlayerController::LMBReleased()
{
	GEngine->AddOnScreenDebugMessage(-1, 1, FColor::Cyan, TEXT("LMBReleased"));
}

then there are multiple scenarios.

You have no widgets or they aren’t handling any of LMB clicks. In this situation every LMB click will fire InputComponent from PlayerController and that will cause to fire LMBPressed() and LMBReleased().
You have widget, that has NativeOnMouseButtonDown() implemented (for LMB) and this function returns FReply::Unhandled(). This cause the bubbling of this event to InputComponent of PlayerController and this led to firing the LMBPressed() and LMBReleased(). The problem is that after NOT handling the widget NativeOnMouseButtonDown() event (but you can do some logic inside that function) you can’t expect to NativeOnMouseButtonUp() to fire (this is, as I understand is made by purpose) and, as a consequence, you can’t do any logic in it.
Your widget returns FReply::Handled() from NativeOnMouseButtonDown(). In this case you can expect to process NativeOnMouseButtonUp() and do some logic in it, but no information about LMB click will be sent to InputComponent of PlayerController.

I’m not asking about how to let know the PlayerController about LMB being clicked using some boolean variable or custom function, but rather how to utilize widget settings or project settings or PlayerController settings to send event about clicks from widget to InputController in other words is there any way in engine to prevent stopping bubbling of events from widgets after they were handled?

and-rad · October 6, 2021, 9:23am

I can’t think of a solution that is as generic as that, sorry. I don’t want to say there is none because someone else might know more about it than I do. Hopefully they can chime in.

If it is any help with regards to the original post, a project I worked on in the past handled selection rectangle inputs entirely in the player controller that just sent the rectangle coordinates to a custom HUD class. The HUD was only responsible for drawing, the logic happened only in the PC, so I never had to deal with conflicting input between UMG and the rest of the game.