Does anyone have a solution to make WidgetComponent in "screen" space interactive?

Hello,

It is my first post here ! I really start using UE4 since 3 months ago, and until now I have found a looot of responses in this forum !
So first: thanks to all people who have struggled with problems before me :smiley: and special thank for people who try to resolved it!

I’m dealing with a bug (which at least seems to be) about WidgetComponent in screen space for mobile device.
I post a short video here to show more details of my problem.

But a short resume here:

  1. I start fresh with a project base on SideScroller for mobile with the Starter Content
  2. I create a UMG widget (call it TestWidget) with a slate button as a child
  3. I create an Actor and attach a WidgetComponent to it
  4. I use TestWidget in the WidgetComponent and set the parameter space to screen
  5. I try to bind functions to differents events (on WidgetComponent, on UMG Widget and on the button)
  6. But no event is triggered on mobile (and when i use Use Mouse for Touch setting too on PC, but only in Select Viewport preview…) :frowning:
  7. I success to catch OnInputTouchBegin event when space setting is set to World, but only the one triggered by the WidgetComponent
  8. (More details in video above)

Seems to be similar to this post, but I have no problems with click event, only touch events cause problems (And I try to set a z-order of 1000 to be sure…):

I’m pretty sure it is related to this bug but unfortunately it seems to be froze:

I’ve created a repo here GitHub - NansPellicari/UE4WidgetCompScreenMobile: A repo to help people understand my problems if you want to test. It can be long to download because I let the Starter Content in (For simplicity reason).

Thank again and I hope my description is clear enough!

Cheers, Nans

For people who struggled with the same issue, I found a solution to this problem, maybe not the best but it’s effective.

These posts helps me to find out:

We override WidgetComponent, in it we can catch the InputTouchState from the controller, check position of the finger and compare to the position and size of the widget:

Thanks to a bunch or vars and conditions we can process touches behaviors:

276726-solution2.png

And it works like a charm!

To go further I will made it in c++ and:

  • override UMG widget to create some functions (eg. DelegateToInputTouchStart, DelegateToInputTouchEnd and DelegateToInputTouchMove for example) which will be called by this WidgetComponent (instead of the Print String nodes you can see here) and are intended to delegate to their respective native event.
  • and create a static var (or maybe in a GameInstance?) to save every widgets currently in screen to manage overlapping

Hope it helps!

ps: This solution in the repo > branch solution1

EDIT

Should be great to check if touch position hit only slate children with a visibility setting to visible.
I’am doing the c++ now, I will post the solution I found soon.

Ok here the solution I found in c++.
This is surrely improvable but it does the job for me.
I have not implemented a way to manage overlapping, it is not necessary for me now.

This code is owning by the UWidgetComponent I mentionned above

file MyWidgetComponent.h

public:
    virtual void TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction) override;

    /// In my own code, this is elsewhere in a library for widgets. I put it here for simplicity sake.
    // This is pasted from UWidgetTree::ForWidgetAndChildren function,
    // But here it allows to go recursively or not
    static void ForWidgetAndChildren(UWidget* Widget, TFunctionRef<void(UWidget*)> Predicate, bool Recursive = true);

private:
    bool bWidgetIsTouched = false;
    FVector2D PrevTouchPos;
    FVector2D TouchPos;
    void ManageTouchEvent();
    bool InWidgetZone(FVector2D TouchPos, UWidget* Widget);

file MyWidgetComponent.cpp

///...
/// associated includes
#include "Blueprint/WidgetTree.h"
#include "Components/Button.h"
#include "Components/CanvasPanelSlot.h"
#include "ExtendUserWidget.h"
#include "Engine/GameViewportClient.h"
#include "Engine/Engine.h"
#include "Slate/SceneViewport.h"

/// ...

void UMyWidgetComponent::TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction * ThisTickFunction)
{
    Super::TickComponent(DeltaTime, TickType, ThisTickFunction);

    // No needs to interact if not visible
    if (!bVisible) return;

    ManageTouchEvents();
}

void UMyWidgetComponent::ManageTouchEvents()
{
    APlayerController* Controller = GetWorld()->GetFirstPlayerController();

    if (Widget == nullptr) return;
    if (Controller == nullptr) return;
    if (!GEngine || !GEngine->GameViewport || !GEngine->GameViewport->GetGameViewport()) return;

    float TouchX, TouchY;
    bool bIsCurrentlyPressed;
    Controller->GetInputTouchState(ETouchIndex::Touch1, TouchX, TouchY, bIsCurrentlyPressed);

    if (!bIsCurrentlyPressed && bWidgetIsTouched) {
        bWidgetIsTouched = false;
        UE_LOG(LogTemp, Warning, TEXT("Touch End"));
        // Final goal is here
        // DispatchOnInputTouchEnd(ETouchIndex::Touch1);
    }

    if (!bIsCurrentlyPressed) return;

    PrevTouchPos = TouchPos;

    TouchPos.X = TouchX;
    TouchPos.Y = TouchY;

    // No interaction here!
    if (Widget->GetVisibility() != ESlateVisibility::SelfHitTestInvisible
        && Widget->GetVisibility() != ESlateVisibility::Visible) {
        return;
    }

    if (!InWidgetZone(TouchPos, Widget->GetRootWidget())) return;

    if (!bWidgetIsTouched) {
        bWidgetIsTouched = true;
        UE_LOG(LogTemp, Warning, TEXT("Touch Start"));
        // Final goal is here
        //DispatchOnInputTouchBegin(ETouchIndex::Touch1);
        return;
    }

    if (!PrevTouchPos.Equals(TouchPos, 0.0001f)) {
        UE_LOG(LogTemp, Warning, TEXT("Touch Move"));
    }
}

bool UMyWidgetComponent::InWidgetZone(FVector2D TouchPos, UUserWidget * Widget)
{
    // No need to go further here
    if (Widget->GetVisibility() != ESlateVisibility::SelfHitTestInvisible
        && Widget->GetVisibility() != ESlateVisibility::Visible) {
        return false;
    }

    // Check if in zone
    if (Widget->GetVisibility() == ESlateVisibility::Visible) {
        FGeometry ViewportGeo = GEngine->GameViewport->GetGameViewport()->GetCachedGeometry();
        // This works only for a visible widget.
        FGeometry Geo = Widget->GetCachedGeometry();
        FVector2D WidgetSize = Geo.GetAbsoluteSize();
        FVector2D WidgetPos = Geo.GetAbsolutePositionAtCoordinates(FVector2D(0, 0));
        WidgetPos = ViewportGeo.AbsoluteToLocal(WidgetPos);

        // Check if touch position is in the widget zone
        if (TouchPos > WidgetPos && TouchPos < WidgetPos + WidgetSize) {
            return true;
        }
    }

    // If touch position is not on the widget or his visibility is set to SelfHitTestInvisible,
    // maybe we have better chance in it's children ?

    TArray<UWidget*> AllWidgets;
    // Retrieve each direct children to test them
    UMyWidgetComponent::ForWidgetAndChildren(Widget, [&AllWidgets](UWidget* ChildWidget) {
        AllWidgets.Add(ChildWidget);
    }, false);

    for (UWidget* SubWidget : AllWidgets)
    {
        // Go to check recursively in this widget's children
        bool bIsInZone = InWidgetZone(TouchPos, SubWidget);

        // If a child is hit, no need to go further
        if (bIsInZone) {
            return true;
        } else {
            continue;
        }
    }

    return false;
}

void UMyWidgetComponent::ForWidgetAndChildren(UWidget * Widget, TFunctionRef<void(UWidget*)> Predicate, bool Recursive)
{
    // Search for any named slot with content that we need to dive into.
    if (INamedSlotInterface* NamedSlotHost = Cast<INamedSlotInterface>(Widget))
    {
        TArray<FName> SlotNames;
        NamedSlotHost->GetSlotNames(SlotNames);

        for (FName SlotName : SlotNames)
        {
            if (UWidget* SlotContent = NamedSlotHost->GetContentForSlot(SlotName))
            {
                Predicate(SlotContent);

                if (Recursive) {
                    ForWidgetAndChildren(SlotContent, Predicate);
                }
            }
        }
    }

    // Search standard children.
    if (UPanelWidget* PanelParent = Cast<UPanelWidget>(Widget))
    {
        for (int32 ChildIndex = 0; ChildIndex < PanelParent->GetChildrenCount(); ChildIndex++)
        {
            if (UWidget* ChildWidget = PanelParent->GetChildAt(ChildIndex))
            {
                Predicate(ChildWidget);

                if (Recursive) {
                    ForWidgetAndChildren(ChildWidget, Predicate);
                }
            }
        }
    }
}