Gamepad Menu Navigation Issues

[tehhax (flub)]

I am trying to include full gamepad (XBox one controller) support. I would like to have a cursor controlled by the left thumbstick and the “a” button (gamepad face bottom) to select (generate a click). Very similar to Destiny’s menu system if you are familiar with the game. I have just set my controls in blueprint to control the mouse cursor with the thumbstick relatively easily. I thought that would be the hard part but apparently generating a click is the real issue.

This is a very similar issue:

Simulate left click with Abutton of gamepad

In the potential solution above, the line trace will not fire the on pressed event of the button. Even if the line trace correctly finds the button, how do I go about “pressing” it with a reference to the object?

Here are some other potential solutions I would like to pass by the unreal experts here:

1. Can I get mouse position, get current visible widget element underneath it, cast to button and somehow fire the onpressed?

2. Another option is on mouse hover to set the current control. Then Gamepad face bottom (A) will fire the onpressed event on the current control.

3. Can I expose C++ code to generate a mouseclick? I exposed the SetMouse() function for gamepad mouse movement but I’ve thoroughly combed through the list of functions and couldn’t see anything that looked like it would generate a click.

The biggest issue here is that I can’t manually fire off the onpressed event in any way other than an actual click or touch to my knowledge.

Another issue:
Mouse controls will be killed if/when this game is exported to Xbox, correct? Xbox does not have mouse support? I can absolutely emulate a “virtual” mouse with some interface work but this is something I would like to consider when picking a solution.

Please help me out!!! This is hurting my brain and I will liberally be handing out +1s to anyone can offer constructive ideas or solutions.

Thank you so much in advance!

[LemIstIth]

You… kinda can, but it’s a little wonky. Here’s what I did:

I ran into this issue and came up with a way to do it, but honestly, it’s probably just one of those things that isn’t grammatically incorrect, but that you should probably just try to avoid doing things that way? (My use is on a menu for a game that will use mouse and keyboard, as well as controller. I used Rama’s plugin to move the cursor, but couldn’t click, (I will explain how I now click) but even after being able to click, it isn’t better than just having an array of buttons, and swapping focus between them with D-Pad or something, and having A activate the highlighted button?

The workaround I came up with (Hopefully images are simple enough to get into this, I’ve never really posted on these forums much/at all before.) involves grabbing button input on the player pawn, which sets a bool. (This could probably be done on a player controller, or any blueprint accepting input if it’s single-player. I have it on the player pawn because I’m using an in-level menu, and am now thinking about moving it to the controller in case of other pawns.

So the pressed output of the input button event for Gamepad Face Button Bottom (A on Xbox, X on PS) sets the variable “IsPressingA” on the player to true. Releasing that button then sets the variable to false.

Then, in the widget blueprint, have an event tick that feeds into a cast to the player (I cast to the owner of the widget, which I assume is probably what you want depending on your use) you want input from (Who has the “IsPressingA” bool on them.) If the cast is works, have it feed into a Branch, which is directed by getting the IsPressingA bool. (Which you can rename obviously.) If true, then you can run that into a sequence with one output for each of your buttons. These outputs then feed into more branches, which will check if each button is currently hovered. (Unless they’re overlapping, this should only ever be true of one of them.) Now, you can feed that “True” branch into the code you have tied to that button, be it a widget swap, or a level change, or exiting the game, or whatever.

Now… you kind of have a way to spoof input. Yay.

Hopefully this is kinda what you were looking for?

[anonymous_user_ab46829d]

Hi,
I was trying this setup and it kind of works but I have one problem. If I use the mouse to click one of the buttons and then try the gamepad it will still hover over the right button but if I press (doesn’t matter where) it will always register the last button I clicked with the mouse. I don’t really understand it. Did you have any problems like that?

[LemIstIth]

I can’t say I have. I’d be curious where the issue is coming from.

The only issues I’d run into so far in using both keyboard AND mouse, and alternating was in cursor movement, which took another layer of swapping a bool on and off depending on if there was gamepad input, then the game would swap into “controller mode” and move the cursor, otherwise just leaving it to the mouse.
(For a long time either worked, but once you used the gamepad, the mouse would stop working, and constantly rubber-band back into it’s original position, because the game was reading the fake mouse X and Y replication variables that I was using to plug into Rama’s victory panel for setting cursor position.)

It sounds like it isn’t picking up on hover events? I also still have a very simple menu, so there could be problems with this method that I simply haven’t run into yet.

If I understand your issue correctly, it sounds like maybe the first click is somehow preventing cursor location tracking for future gamepad inputs? I’d make sure everything can tick while paused, etc. I ended up weighing issues with a user using BOTH higher, in retrospect, than I probably should have.

Although, the way you describe it makes it sound like the cursor is still tracking position/moving, which would make me think it’s just an issue with how widgets work. I’d try re-setting or re-drawing the widget after each button press I guess? See if that clears it up?

I’d love to help if at all possible though, I’ll keep an eye on this thread.

The key points being:

• Sorry it isn’t working.

• I’d check that everything involved runs/checks input while paused.

• I’d make sure it checks which button is being hovered/pressed every time there’s input.

• You may want to re-draw the widget every time you push a button. (I’ve had some issues with stuff not refreshing or breaking if the menu isn’t re-drawn in the past. (Although, not with this yet.))

[EstherZuidgeest]

I found a solution for the left click event that triggers UI ‘on clicked’ events.
I’ve looked into this because I want a handtracker to move the mouse and also click things.
You’ll have to go into c++ to make it work, but its not too complicated and if you struggle with it, let me know.
So add a new c++ class which inherits from player controller (this is nog mandatory but its nice to have it in a player controller) Then in your .h file add 2 functions like so:

public: 	
    UFUNCTION(BlueprintCallable) 	
    void TriggerMouseLMBDown();

    UFUNCTION(BlueprintCallable) 	
    void TriggerMouseLMBUp();

The blueprint callable makes it so that you can trigger the function from blueprints.
The first function simulates the click down and the second function the release. You need both.

Then in the .cpp file add the code below.
The first part triggers any LMB click events and the part below that handles the UI clicks.

void AHandTrack_PlayerController::TriggerMouseLMBDown()
{
	//trigger the mouse click event. This will fire any lmb click events within blueprints.
	FViewportClient* Client = GEngine->GameViewport->Viewport->GetClient();
	FKey MouseLMB = EKeys::LeftMouseButton;
	Client->InputKey(GEngine->GameViewport->Viewport, 0, MouseLMB, EInputEvent::IE_Pressed);

        //Trigger mouse clicks in UI
	//Get our slate application
	FSlateApplication& SlateApp = FSlateApplication::Get();

	//create a pointer event
	FPointerEvent MouseDownEvent(
		0,
		SlateApp.CursorPointerIndex,
		SlateApp.GetCursorPos(),
		SlateApp.GetLastCursorPos(),
		SlateApp.GetPressedMouseButtons(),
		EKeys::LeftMouseButton,
		0,
		SlateApp.GetPlatformApplication()->GetModifierKeys()
	);
	
	//send the mouse event to the slate handler
	TSharedPtr<FGenericWindow> GenWindow;
	SlateApp.ProcessMouseButtonDownEvent(GenWindow, MouseDownEvent);
	
}

void AHandTrack_PlayerController::TriggerMouseLMBUp()
{
	//trigger the mouse click release event
	FViewportClient* Client = GEngine->GameViewport->Viewport->GetClient();
	FKey MouseLMB = EKeys::LeftMouseButton;
	Client->InputKey(GEngine->GameViewport->Viewport, 0, MouseLMB, EInputEvent::IE_Released);

	//trigger the UI mouse click
	FSlateApplication& SlateApp = FSlateApplication::Get();

	FPointerEvent MouseUpEvent(
		0,
		SlateApp.CursorPointerIndex,
		SlateApp.GetCursorPos(),
		SlateApp.GetLastCursorPos(),
		SlateApp.GetPressedMouseButtons(),
		EKeys::LeftMouseButton,
		0,
		SlateApp.GetPlatformApplication()->GetModifierKeys()
	);

	TSharedPtr<FGenericWindow> GenWindow;
	SlateApp.ProcessMouseButtonUpEvent(MouseUpEvent);
}

For my hand tracking solution in the blueprint I trigger the mouse click down on a hand pinch.
Don’t forget to also trigger the mouse click up events with a small delay or on a button release.
In your instance you can detect a specific button press and then trigger the mouse click down function and trigger the mouse click up on button release.

[tehhax (flub)]

Very cool! Wish I had this solution when I was dealing with the problem over 3 years ago haha. Appreciate you posting the answer so long after the fact so it may help people in the future. Upvoting the answer and if someone else confirms it works I’ll accept the answer

[Nasti_N8]

This answer didn’t totally work for me, but I was able to modify it to get it working with widgets.

Simulate any mouse button click Unreal Engine 5.2 c++ - Pastebin.com