I am not an Unreal dev, but a devops guy. I have to automate the startup of UE4 pixel streaming.
Currently the following are started manually by a logged in user and everything works fine:
UE4/pixel streaming
WebRTC
Signalling server
Once all up and running I can navigate to the URL and see the pixel streamed image and can use my mouse in the viewer widget to spin around the character. All works as expected.
I then automated the startup so that the above are started from a Windows scheduled task without a user logged in.
However, when I navigate to my browser everything works EXCEPT I cannot use the mouse to spin around the character. The mouse scroll wheel does allow me to zoom in and out. I can use the mouse to click the buttons on the page (outside the viewer widget).
I can sometimes spin very slightly up and down in a jerky way in the viewer widget - but its very hit and miss.
It looks like the mouse events are sent to UE4, but UE4 does not act on them correctly.
My question is whether UE4 is supported to be started and run by a scheduled task. Or does it require that a user is actively logged in.
Any ideas on what may be wrong? It is such an odd issue. If I stop just UE4 start manually then it works again. If I restart once again with a Windows Scheduled task then it has the same weird issue.
My colleague solved the issue. We had to start with:
PsExec.exe -d -s -i 1 game.bat
Where game.bat called game.exe (ie the packaged game) with its various arguments.
Just trying to use start-process or similar ways failed as the game ran in some sort of non-interactive mode when not started directly by a user which seems to be some weird Windows issue.
Hello!
It seems you have encountered an issue that my team is dealing with. After speaking with Epic, they are looking into it, but I was wondering how you implemented your solution.
How did you get the Task Scheduler to start the game.bat with the arguments to psexec?
I gave that a shot but still no success with a working mouse. I am able to run a PSexec command with my username and password manually and it works with a mouse, but when I run via the task scheduler or services, the mouse only works with HUD items, not objects in the viewport.
Are there any other particualr configs you are doing for the task scheduler?
Sorry I forgot about that part of the OP. My use case is actually different, I’m just running a project on an EC2 instance when there’s no user logged in. The point of using PsExec was to run it “remotely” under the system account, which I’m pretty sure will make using a mouse not work.
In order to receive mouse input you’ll probably need to run it under an account that’s logged in and has the window visible. Windows typically only sends input to the focused window. You can probably modify the C++ though to plug into User32.dll and get the mouse events that way.
What I was suggesting with using User32.dll or more accurately the Windows API is way less elegant, but essentially you’d be making your own code to get the mouse position and click events. Like this: mouse_event function (winuser.h) - Win32 apps | Microsoft Docs
From that you can use the mouse position (dx and dy), subtract the window position, then deproject the position into world space using APlayerController::DeprojectScreenPositionToWorld. That can give you a ray that you can use to ray cast any object in the scene.
I wouldn’t do that if you have other methods you can think of though, that’s more of a last resort idea.