Hello, I’m attempting to use AutomationDriver to test some of our custom Slate UI. I am encountering an issue with getting consistent passing tests in RDP/headless mode due to what I believe might be a bug with the AutomationDriver implementation. I am on Unreal 5.3 buildingfrom source, but I backported a partial fix from 5.4:
I’m opening this thread to provide an explanation for this issue and my fix. However, most importantly I’d like a confirmation from a subject matter expert on whether this is actually a bug or if there’s some intended behavior here that I’m pushing against.
FAutomatedApplicationImpl::IsCursorDirectlyOverSlateWindow() and GetWindowUnderCursor() delegate to RealApplication when bAllowMessageHandling is true (the default after Enable()). The real application queries the OS for the window under the physical cursor position, not the automated cursor position set by the driver. On RDP or headless environments where the physical cursor is offscreen or at (0,0), these methods report no window under the cursor. Slate’s input processing then discards the simulated mouse events, causing all click/hover interactions via the AutomationDriver to fail.
The fix is for both methods to always use InternalGetWindowUnderCursor(), which checks visible windows against AutomatedCursor->GetPosition(), rather than conditionally delegating to the native OS API.
The 5.4 fix addressed GetPosition() (via bOverrideRealCursor in StepExecutor) and GetModifierKeys() (condition inversion). That fix did not cover IsCursorDirectlyOverSlateWindow() or GetWindowUnderCursor() because they bypass FAutomatedCursor entirely. They call native OS APIs that don’t go through GetPosition(), so bOverrideRealCursor has no effect on them.
Separately, Enable() should call UsePlatformCursorForCursorUser(true) immediately after SetPlatformApplication() to force FSlateUser to pick up the FAutomatedCursor. Without this, FSlateUser retains the old FWindowsCursor until 3+ platform mouse-move events trigger the auto-switch (SlateApplication.cpp ~line 5933), causing intermittent test failures. The 5.4 fix added this call to Disable() for cleanup between test runs but not to Enable() for initialization.
When all of these changes are made, I can successfully pass all of my Slate AutomationDriver tests in:
- SessionFrontend
- RDP
- Rider’s Unit Test driver
Additionally, I can move my physical mouse as much as I want during the test and there is no impact to the run. This is a particularly big boost to the utility of these tests as it reduces flakiness and doesn’t interrupt devs who may want to be actually using their mouse during test runs.