Emulating Mouse Clicks with Win32 API

Development Programming & Scripting
,
1

I’m building an application in Unreal 5.3 with UMG, where the user inputs are coming from a secondary device. This second device is capturing mouse movements from a Human Interface Device, and I’m sending JSON packets over UDP with the data:

{
    "data" : {
        "x" : 0,
        "y": 0,
        "mouseDown" : True
     }
}

These JSON packets are streamed continuously, and on the receiving end a python script parses them and runs:

    win32api.SetCursorPos((data["x"], data["y"]))
    if data["mouseDown"]:
        win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN, data["x"], data["y"], 0, 0)
    else:
        win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP, data["x"], data["y"], 0, 0)

This script works great and allow me to pass through cursor movements and mouse clicks that work with all windows applications EXCEPT my Unreal Application. The cursor moves, but when my Unreal Application is in focus, the clicks do absolutely nothing in the application. After extensive troubleshooting, I am convinced that this is an issue within my application.

Does Unreal Engine natively ignore “virtual” mouse events and require that they originate from a HID driver? And if so, are there easy configuration settings that would allow for emulated mouse events? I’ve dug around in settings but can’t find anything that I think would enable / disable my emulated mouse events.

I will add that when using my actual mouse, the application works fine – so it really does seem specific to these emulated events and not ones originating from hardware devices.

2

Well, a minor thing to check first in such kind of questions: do you have any delay between DOWN & UP or are they sent in a single UE’s frame? Some engines won’t detect anything if there was no delay between them

3

Right now the events are being forwarded from an actual mouse that’s being clicked on the secondary device. I’m assuming that because they’re being forwarded, the timing is similar to the original timing, and if it would qualify as a click on secondary computer it should work in my application as well.

I suppose that it is possible that once I’m sending the UDP messages, they’re arriving at slightly different time intervals and could arrive too close.

That’s sad, do you UMG buttons respond to the mouse down, mouse up, or do they actually do a edge detection between the two states?