What is the best way to toggle a mouse-driven UI overlay?

So I’m working on an in-game tool to visualise certain gameplay statistics.
I’ve got a UI based on SCompoundWidget that I’m adding with AddViewportWidgetContent and currently using ClearKeyboardFocus() and ResetToDefaultInputSettings() to get a cursor that will automatically focus on the widget (which I am unsure is the best way to do so, but seems to work well enough for now, based on what shift+F1 does).

What’s the best way to push my Slate system to the input stack so that all input is forwarded to my UI when I show it, then be able to pop it back off the stack at the same time as hiding it?

I’m currently working in ShooterGame, but this system is designed to be game-agnostic so I would prefer not to have to subclass PlayerController or related classes.

There are many ways to do it, but they all come down to

  1. Instantiate a new widget and add it to the live widget tree (using AddViewportWidgetContent() is one way to do it)
  2. Set focus to that widget (do this AFTER you have added it to the live widget tree). Your widget must be focusable, so you will need to override SupportsKeyboardFocus().
  3. When you are done with the widget, set focus back to game viewport and remove your widget from the live widget tree.

This actually involved a little bit of trickery depending on what behavior you want from the UI. I am planning to work on API improvements for doing exactly this sort of thing toward the end of August (tentatively; timing might change).

EDIT: For the sake of removing ambiguity, I’ll make an example.

Ideally, the Viewport would support this with a single switch. However, it does not right now. Here is how you would work around it.

Presumably, you press some button (let’s say that it is [M]) to go from CameraMode to CursorMode.
In CursorMode you want to show a widget called SItemManager.
Your SItemManager widget will be where the user can use the mouse to manage some item widgets.
You can add your SItemManager widget into the game with AddViewportWidgetContent().

Going into CursorMode will amount to setting focus on the SItemManager; you can do this via FSlateApplication.Get().SetKeyboardFocus(). Your SItemManager widget will be notified when it has been focused, and this event is your opportunity to make appropriate changes to appear in CursorMode. For example, you’ll want to release mouse capture. You’ll also want to show some of the widgets used for managing the items.

When you want to dismiss this screen, you will just set focus back to the viewport. The viewport will aggressively modify the cursor state back to what is needed for camera mode. This is why we cache the WidgetFocusedBeforeMe upon being focused. That widget will be the viewport.

class SItemManager : public SCompoundWidget
{
public:
	SLATE_BEGIN_ARGS( SItemManager )
	{}
	SLATE_END_ARGS()

	void Construct(const FArguments& InArgs)
	{
		this->ChildSlot
		[
			// My Child Widgets
		];
	}

	virtual void ToggleCursorMode()
	{
		if (bInCursorMode)
		{
			bInCursorMode = false;
			FSlateApplication::Get().SetKeyboardFocus(WidgetFocusedBeforeMe);
			WidgetFocusedBeforeMe.Reset();
		}
		else
		{
			// ORDER of statements MATTERS!
			// Check OnKeyboardFocusReceived and SupportsKeyboardFocus before moving these statements.
			bInCursorMode = true;
			WidgetFocusedBeforeMe = FSlateApplication::Get().GetKeyboardFocusedWidget();
			FSlateApplication::Get().SetKeyboardFocus(SharedThis(this), EKeyboardFocusCause::Keyboard);
		}

	}

	virtual FReply ExitCursorMode()
	{
		if (bInCursorMode)
		{
			ToggleCursorMode();
		}

		return FReply::Handled().EndDragDrop();
	}

private:

	// BEGIN SWidget Interface

	virtual bool SupportsKeyboardFocus() const OVERRIDE
	{
		return bInCursorMode;
	}

	virtual FReply OnKeyboardFocusReceived(
		const FGeometry& MyGeometry,
	    const FKeyboardFocusEvent& InKeyboardFocusEvent ) OVERRIDE
	{
		// HACK: Work-around for held keys.
		GetPlayerController()->FlushPlayerInput();

		const bool bSwitchingToCursorMode = InKeyboardFocusEvent.GetCause() == EKeyboardFocusCause::Keyboard;
		if (bSwitchingToCursorMode)
		{
			// We are switching to CursorMode...
			return FReply::Handled()
				// ... mouse input was captive by the game viewport
				// mouse needs to be free to interact with menus
				.ReleaseMouseCapture()
				// Joystick should not control player while menus are up
				.ReleaseJoystickCapture()
				// The mouse should still not escape the boundaries of the game.
				.LockMouseToWidget( SharedThis(this) );
		}
		else
		{
			// Upon focusing we were already in CursorMode, and the mouse should
			// move freely around the desktop until the player clicks on the game.
			return FReply::Handled().ReleaseMouseCapture().ReleaseJoystickCapture();
		}
	}

	virtual FReply OnKeyDown(
		const FGeometry& MyGeometry,
	    const FKeyboardEvent& InKeyboardEvent ) OVERRIDE
	{
		const EKey PressedKey = InKeyboardEvent.GetKey();

		if(PressedKey == EKeys::Escape)
		{
			return ExitCursorMode();
		}
		else if (PressedKey != EKeys::Tilde)
		{
			// Incercept keyboard control; we do not want camera/character control
			// while in extended HUD.
			// Make an exception for system level keys, e.g., Tilde;
			return FReply::Handled();
		}
		else
		{
			return FReply::Unhandled();
		}

	}

	virtual FReply OnPreviewMouseButtonDown(
		const FGeometry& MyGeometry,
	    const FPointerEvent& MouseEvent ) OVERRIDE
	{
		// When we are in cursor mode, clicking on the window should lock the cursor to the window
		// but should not alter anything the click would have done (e.g. press a button).
		// This preview event accomplishes that.

		if (bInCursorMode)
		{
			return
				// Notice we have NOT handled the event!
				FReply::Unhandled()
				// Just need to force lock the mouse to the viewport
				// area without affecting any other interaction.
				.LockMouseToWidget(SharedThis(this));
		}
		else
		{
			return FReply::Unhandled();
		}
	}

	// END SWidget Interface

	/** In cursor mode we show an extended, interactive HUD and   */
	bool bInCursorMode;
	/**
	 * We try to be a good citizen, and restore focus to whoever had focus before we did.
	 * In this case it is probably the game viewport.
	 */
	TSharedPtr<SWidget> WidgetFocusedBeforeMe;

};

Let me know if you run into trouble with implementing this.

This works fine for setting the focus to the widget, Nick, thanks.
However, when I call ExitCursorMode the cursor remains visible and focus does not shift back to the viewport. Do you have any further suggestions?

Is the focus being set back to the SViewport widget? If so, I would put a breakpoint in SViewport::OnKeyboardFocusReceived and/or FSceneViewport::OnKeyboardFocusReceived. You want to make sure that the FSceneViewport version of the function is getting hit and returns an FReply that would set the Viewport state correctly. Obviously this all depends on WidgetFocusedBeforeMe getting set up to point at the SViewport, but I believe that should be working correctly.

Please try those breakpoints and let me know what you find.

Both of those functions are being hit. What are the key properties of the return value from the FSceneViewport version should I be looking for? the FReply it is returning has the following properties:

MouseCaptor and JoystickCaptor set to the SViewport

EventHandler, FocusRecipient set to null

bIsHandled 1

bReleaseMouseCapture 0

bReleaseJoystickCapture 0

bAllJoysticks 1

bShouldReleaseMouseLock 0

bUseHighPrecisionMouse 1

bPreventThrottling 0

bEndDragDrop 0

FocusChangeReason SetDirectly

You could be doing something else to make the cursor visible. When you say that the “focus doesn’t shift back to the viewport”. Do you mean that your keyboard and mouse do control the camera?

Cursor visibility is determined by the OnCursorQuery() call. The SViewport has one of those, and it forwards it. You can set a breakpoint in there to figure out what type of cursor it is returning. EMouseCursor::None will hide the cursor when the mouse is over a widget. That is what SViewport will return to make the cursor disappear. However, it delegates the logic to other code; you can step through to see what is being returned.

At the moment I have an SButton which calls ExitCursorMode. When the button is clicked, the cursor becomes locked to the viewport (because of LockToWidget), but moving it does not control the camera, and child widgets in my UI still show mouse-over effects, such as SListView highlighting the background of child entries in the list, SButtons highlighting when the cursor is over them, too, which leads me to believe my widget still has focus at that point, especially given that I can click on widgets as per normal. The keyboard is able to control the player’s movement. At this stage clicking on an area not populated by UI elements restores mouse control to the player’s camera, and hides the cursor.

I’ve identified the cursor’s visibility as being due to my use of .Cursor() on my widget when I created it, but removing that part of widget initialization means that the cursor disappears when I click the button, but I still need to move my now-invisible cursor back to an empty location of the screen and click in order for mouse movement to control the camera.

The SViewport needs to gain mouse capture in order to control the camera. It sounds like this isn’t happening for whatever reason. However, setting focus on the SViewport should cause it to acquire mouse capture.

It’s rather difficult to debug this via answer hub. I hope to have a dramatically easier-to-use API for handling these tasks ready by end of August. If you still need help, I’m in the #unrealengine channel on freenode (http://webchat.freenode.net/). Please be aware. I am on Korean time until ~August 14.

For future reference for anybody, I ended up using

return FReply::Handled().CaptureMouse(FSlateApplication::Get().GetGameViewport().ToSharedRef())
		.CaptureJoystick(FSlateApplication::Get().GetGameViewport().ToSharedRef())
		.LockMouseToWidget(FSlateApplication::Get().GetGameViewport().ToSharedRef())
		.SetKeyboardFocus(FSlateApplication::Get().GetGameViewport().ToSharedRef(), EKeyboardFocusCause::Mouse);
		.UseHighPrecisionMouseMovement(FSlateApplication::Get().GetGameViewport().ToSharedRef());
}

The key line which fixed things was the UseHighPrecisionMouseMovement. Documentation mentions that this ‘implies mouse capture and hidden mouse movement’. Is there any documentation on the implementation of ‘hidden mouse movement’? It sort of feels clunky to use this call, in terms of expressing my intent in code.

Awesome. Glad you figured it out! BTW, we are currently working on making this set of tasks much easier with a higher-level API that expresses intent much more concisely.

Did you ever manage to expose this API further? I’m trying to implement a kind of mouse position lock similar to FPS games in UMG with little luck. Thanks!