I’m experimenting with the pixel streaming plug in in 4.21. I’m testing it locally on separate computers, one runs it and another one is the client. I can start the app and run it in Chrome, no problem. But I can’t use the mouse cursor and therefor I can’t click on any buttons in the app. Once I start playing the cursor disappears and this pop up shows:
When I press ESC the cursor appears but as soon as I click anywhere including on a button it disappears again.
I made a (different) version for touch screen and it works fine on a tablet without a mouse. I can touch the buttons and they will work as expected.
Anyone knows how to use a mouse in a pixel streamed project?
When you play your packaged application locally, is your mouse visible?
The player.html page has a couple of options for mouse cursor handling. The default one, LockedMouse, does what you’re describing. However, normally, after you’ve clicked on the player widget, the cursor doesn’t disappear entirely. It may be that if the cursor isn’t visible locally in your packaged application, it also won’t be visible in the remote viewer widget.
You could also try changing the control scheme to HoveringMouse. In this case the cursor should remain visible, but won’t be locked to the boundaries of the player widget. See the controlScheme option in the docs here:
thanks for your answer. It works fine when I play the packaged application. The mouse cursor is hidden when the player is in full screen mode and all widgets are removed and it shows up again once a widget is on screen. I am using the “Show Mouse Cursor” variable and “Set Input Mode…” blueprint nodes.
It doesn’t translate over to the player web page though. It seems that hiding the mouse cursor is driven in the player.html page and not through the application itself. Anything set in the app will always be overwritten by the player.htm page. Even if I never hide the cursor by removing the blueprint nodes or if I set the “show mouse cursor” variable to be always ticked the cursor always disappears.
Do I have to send a java script to the player.htm from the app to show (and hide) the cursor? I never worked with java. Could you show me how that would be done. I find it hard to understand the documentation about the player web page. I just don’t have any experience with programming in java. Cheers.
I made a custom cursor, a red circle to troubleshoot.
I noticed that the first time you click on the page to play the application, the mouse cursor gets moved to 0,0 which is the top left corner of your screen.
I set the cursor to be in the middle of the screen at begin play, but it’s not reliable. Sometimes it gets reset to 0,0 anyway.
I thunk that confirms that it must be the web page that is resetting the cursor.
is there any new information regarding this problem?
Because the only work around so far has been to use 3d Widgets. It can’t be the case that the 2D UMG UI isn’t useable with Pixelstreaming.
I’ve been looking into this again and it seems the app runs fine now with mouse control. It seems that the app switches between UI and gameplay now. So clicking on a button will trigger that button and clicking and dragging somewhere else will move and rotate the player around the scene. I have a “mouse move” option where a target appears on the floor wherever I point the mouse and clicking will move the player to that target.
What I am struggling with at the moment is using a touch screen interface. I can use the buttons but I can’t move by clicking on the target. There is no widget where the target is, just a decal on the floor. But just touching the screen somewhere could be interpreted as a mouse click?? - Or maybe I have to create an invisible widget that follows the mouse on the screen. Anyone reading this has an idea how to enable pixel streaming on a tablet without using the virtual controls? With some sort of target move similar to VR controls?
I am looking into the html player page now. Maybe the switch between UI and gameplay (for touch) can be triggered there?
Comment out Line 1264, uncomment Line 1265. Should look like this after.
//styleCursor = 'none'; // We will rely on UE4 client's software cursor.
styleCursor = 'default'; // Showing cursor
Optionally, go to Line 629 and change it to this.
controlScheme: ControlSchemeType.HoveringMouse,
Make sure your mouse cursor is set to visible in UE4 itself, and “Hide Cusor During Capture” is unchecked in your “Set Input Mode Game and UI”
If you don’t like the trailing software cursor as well, in your Project Settings under Plugins - PixelStreaming and make sure your “Pixel Streamer Default Cursor Class Name” is set to [None].
Albeit there will be a bit of input lag, but if there is a project in which you need a cursor at all times, this works.
I’m facing the same problem you had right now, but with 4.26.1 … I followed the tutorial and the setup works fine, but the mouse cursor is not visible. Local everything works fine. But through the web player I’m missing the cursor - interaction like clicking somewhere is possible, only it’s quite difficult to navigate.
Did you solve it for your case? Or did it magically work somewhen?
I was using a UI with blueprint nodes that show and hide the mouse cursor. It seemed to be working. That was in 4.25. I tested it on the same PC.
I started the pixel stream server, started the unreal app in ‘hidden’ mode where you can’t see the visuals, started chrome and connected to the IP address and it worked. But there is always the chance because both unreal and web app are running on the same PC that the unreal app actually catches the input directly and not via the web connection. So maybe it didn’t work. But I think it worked as expected. 4.25.
I’ve a HUD like UI, just some buttons and in the player controller the mouse is shown and the game mode is ‘Game and UI’. Nothing fancy with blueprints. So I’d think that should work fine :rolleyes: I tried the Pixel Streaming with ‘Play Standalone’ and with the packaged game, but I’d like to try the ‘Hidden’ mode you mentioned. Therefore I’ve to add -RenderOffScreen to the parameters I guess?
No, that doesn’t work either. With the PixelStreaming Demo it works - so I checked my Widgets and PlayerController settings and even disabled most of it … but the mouse stays hidden. I think I’ve to wait for the plugin to grow. It may work sometime for me!
Try those nodes in your level blueprint or in your character. I connected them to Begin Play and also to pressing 8 on the keyboard just in case. The ‘8’ isn’t necessary. Put the GetPlayerController in first and then drag off the variables from that node.
There must be some problem with my setup anyway. When I use the workaround you suggested and press the ‘8’, not even my mouse won’t show up - I even lock my running game from registering mouse inputs. I can see the cursor, but the clicks won’t work anymore. (Navigating with keyboard works fine)
Kind of strange - but I’m afraid I’ve to dig through the blueprints. :rolleyes:
After tidying up my Player Controller and the Widgets (a whole lot of unnecessary’Show Mouse Cursor’ nodes around :rolleyes:) I tried it again - but nothing changed.
Just before I gave up I checked the Plugin Settings (Inside Project Settings) for PixelStreaming … and there I changed the ‘Pixel Streamer Default Cursor Class Name’ to DefaultCursor.
Before it was on ‘None’. Don’t know if I changed that value somewhen … but now it works fine! And my nodes are a little bit more organized. Win Win. :o
I did not find a solution to my problem in this thread. It seems that for some unknown reason the mouse just does not show up in the browser when setting the mouse to true in game. I fixed it by editing the app.js so that I can press a button to open an inventory screen and have the mouse available. To put in steps what I did:
Moved the playerElement variable out of the onConfig function and into the global variables
Edited the registerhover function by changing styleCursor to ‘default’ and adding an empty playerElement.onclick function
Edit the registerkeyboards event to include a new key in the event:
if (e.code === "KeyI")
{
switch (inputOptions.controlScheme) {
case ControlSchemeType.HoveringMouse:
inputOptions.controlScheme = ControlSchemeType.LockedMouse;
playerElement.requestPointerLock();
registerLockedMouseEvents(playerElement);
break;
case ControlSchemeType.LockedMouse:
inputOptions.controlScheme = ControlSchemeType.HoveringMouse;
document.exitPointerLock();
registerHoveringMouseEvents(playerElement);
break;
default:
console.log(`ERROR: Unknown control scheme ${inputOptions.controlScheme}`);
registerLockedMouseEvents(playerElement);
break;
}
}
This worked a lot better for me as this calls the pointer locks to release the mouse cursor and reenable them when pressing ‘i’ again. The main disadvantage of this is that if the person wants a different key for inventory I have to change it in 2 locations.
The PixelStreamingInfrastructure was updated and they removed the app.js. You might still be able to implement this solution in the KeyboardController, but I am unsure.
With the PixelStreaming Demo it works - so I checked my Widgets and PlayerController settings and even disabled most of it … but the mouse stays hidden. I think I’ve to wait for the plugin to grow. It may work sometime for me!