How to inject/simulate mouse clicks?

Development Programming & Scripting C++
1

Hi guys,

I sucessfully managed to control the mouse cursor in a UMG menu using the analog stick on my PS4 gamepad.

However, what I didn’t think about before setting it all up: How can I inject mouse clicks by pushing buttons on my gamepad?

Does anybody have some pointers for me? Any help appreciated!

Thanks.

2

Subclass FAnalogCursor and register it with FSlateApplication’s SetInputPreProcessor. The FAnalogCursor handles the grunt work of mapping normal stick movements and the bottom face button of the controller to function the same as a mouse, but you can further refine the controls by subclassing it.

1 Like
3

In the meantime I was experimenting with the Windows-internal “SendInput” function. Still have some issues there, but seems like it’s possible that way as well.

Thanks, I’ll look into the classes you mentioned.

4

Could you let me know how you managed to control the cursor in the UMG menu with your gamepad? Cause I’m struggling here haha.

5

When I implemented basic VR gaze input/cursor system, I made a GazeInput plugin using code adapted from SlateApplication’s input processing code. Using this method I was able to push input events into the widget system.

6

I composed a plugin that lets you navigate UMG menus with a gamepad, Nick Darnell wrote the code, buttons for mouse clicks works great in this plugin :slight_smile:

C++ code in the download here!

Gamepad-Friendly UMG ~ Control Cursor with Gamepad Analog Stick! Easily Click Buttons! - Programming & Scripting - Epic Developer Community Forums!

Rama

7

I’ve managed to find how to generate both mouse click events and UI mouse clicks.
Here are two functions i expose to blueprints that will simulate mouse clicks for me.
In my player controller blueprint I trigger mouse down function when i do a specific hand gesture and after a short delay I trigger the mouse up function.

#include “HandTrack_PlayerController.h”
#include “Public/UnrealClient.h”
#include “Classes/Engine/Engine.h”
#include “Classes/Engine/GameViewportClient.h”
#include “InputCoreTypes.h”
#include “SlateApplication.h”

void AHandTrack_PlayerController::TriggerMouseLMBDown()
{
//trigger the mouse click event. This will fire any lmb click events within blueprints.
FViewportClient* Client = GEngine->GameViewport->Viewport->GetClient();
FKey MouseLMB = EKeys::LeftMouseButton;
Client->InputKey(GEngine->GameViewport->Viewport, 0, MouseLMB, EInputEvent::IE_Pressed);

//Get our slate application
FSlateApplication& SlateApp = FSlateApplication::Get();

//create a pointer event
FPointerEvent MouseDownEvent(
    0,
    SlateApp.CursorPointerIndex,
    SlateApp.GetCursorPos(),
    SlateApp.GetLastCursorPos(),
    SlateApp.GetPressedMouseButtons(),
    EKeys::LeftMouseButton,
    0,
    SlateApp.GetPlatformApplication()->GetModifierKeys()
);

//send the mouse event to the slate handler
TSharedPtr<FGenericWindow> GenWindow;
SlateApp.ProcessMouseButtonDownEvent(GenWindow, MouseDownEvent);

}

void AHandTrack_PlayerController::TriggerMouseLMBUp()
{
//trigger the mouse click release event
FViewportClient* Client = GEngine->GameViewport->Viewport->GetClient();
FKey MouseLMB = EKeys::LeftMouseButton;
Client->InputKey(GEngine->GameViewport->Viewport, 0, MouseLMB, EInputEvent::IE_Released);

//trigger the UI mouse click
FSlateApplication& SlateApp = FSlateApplication::Get();

FPointerEvent MouseUpEvent(
    0,
    SlateApp.CursorPointerIndex,
    SlateApp.GetCursorPos(),
    SlateApp.GetLastCursorPos(),
    SlateApp.GetPressedMouseButtons(),
    EKeys::LeftMouseButton,
    0,
    SlateApp.GetPlatformApplication()->GetModifierKeys()
);

TSharedPtr<FGenericWindow> GenWindow;
SlateApp.ProcessMouseButtonUpEvent(MouseUpEvent);

}

5 Likes
8

For anybody that wants a blueprint hacky way in UE 4.22. you could check every tick for which “Is Input Key Down” and then test against a bool if the button is hovered over when the interact button is pressed. In order to get that Up and Down Action only once, use “Do Once” and reset each after the opposite is complete.

This doesn’t inject or simulate a mouse click so much as it allows you to get the gamepad input within a Widget blueprint and interact with things only on hover.

The second image is the same code, but as a macro and includes the option to execute if the key is held/pressed.

1 Like
9

thank you!
this seems to be the solution for anyone who already implemented the movement themself (like I also did) before finding out there’s a solution for this built into Unreal but kind of hidden, called FAnalogCursor, as mentioned above.
documentation on FAnalogCursor seems lacking.
I know “the code is the documentation” but still.

10

GamepadHUD2
GamepadHUD21676×736 110 KB

GamepadHUD1
GamepadHUD11890×1264 317 KB

It’s not the greatest solution, and is quite tedious depending on how much you have in your HUD, but this works for me at least. I found I needed to overlay my non-btn UMG elements that had click events with a transparent btn, that way you can use the OnHovered and Unhovered events to set a boolean. Then you can simply collapse all your click logic into a function, and call that from your pawn/character’s BP.

Definitely not perfect, but while I was quite keen on trying Ramu/Nick’s plugin, it’s not updated for UE5 yet from what I saw on Ramu’s YT channel. So in the meantime this is some sort of solution I guess lol

12

I was able to modify this to get it working correctly with widgets.

1 Like
13

Not working, but this one is:

    UFUNCTION(BlueprintCallable, Category = "Utility")
        static void SimulateLeftMouseButtonClick(bool bButtonDown)
    {
        FSlateApplication& SlateApp = FSlateApplication::Get();
        FPointerEvent MouseEvent(
            0, 
            SlateApp.CursorPointerIndex,
            SlateApp.GetCursorPos(),
            SlateApp.GetLastCursorPos(),
            SlateApp.GetPressedMouseButtons(),
            EKeys::LeftMouseButton, 
            0, 
            SlateApp.GetPlatformApplication()->GetModifierKeys()
        );

        if (bButtonDown)
        {
            SlateApp.ProcessMouseButtonDownEvent(nullptr, MouseEvent);
        }
        else
        {
            SlateApp.ProcessMouseButtonUpEvent(MouseEvent);
        }
    }

Thank me later.

1 Like
14

Thanks for sharing. this works in UE5.3

1 Like