I’m creating a co-operative VR experience, with one player in VR and the other sitting at the computer. I need the desktop display to have a different camera view, with interactive / clickable UMG elements in it.
I have set up the camera - a Scene Capture 2D in the scene, linked to a Render Target which is then fed into the Spectator Screen Texture with Spectator Screen Mode set to ‘Texture’. That all works.
I have also enabled the mouse by setting ‘Enable Click Events’, ‘Enable Mouse Over Events’, and ‘Show Mouse Cursor’ to true.
So when I run the game in VR, I have the normal view in the HMD, and my custom camera view showing on the computer monitor, with the mouse cursor showing. Perfect.
However, I cannot figure out how to add UMG widgets to the spectator view. If I ‘Create Widget’ and ‘Add to Viewport’, it does not appear.
Anyone know how to make this work?
There is a line on this page which says: “the Texture can serve as a render target (which you can write to from a SceneCaptureComponent2D), or a container for User Interface elements…”
So it sounds like it’s theoretically possible to put UMG widgets in a texture and project it here, but does that mean that the UMG elements wouldn’t be interactive, ie. respond to mouse-over events and clicks?
I was literally about to post a very similar question. I do not know the answer but I would one if you have ANOTHER player controller and try that route? I’ll be experimenting…
The tricky part is, even if you get the UMG to work, moving the mouse would normally move the player’s viewport, so that has to be address.
Setting all those mouse bools to true seems to mean that the mouse doesn’t affect the player’s viewport.
I’m not sure how you’d go about having another player controller rendering it’s camera to the desktop viewport while the VR player just renders to HMD - is that possible? If it’s possible to do it like that, then I imagine you could probably render UMG into it as normal.
Would be much simpler if it was possible to just add UMG to the VR spectator window though, and it’s not clear to my why that wouldn’t be possible. Perhaps an Epic VR dev can weigh in here…?
Hi there,
That’s kinda interresting, any infos from Epic would be fine… It’s written in the doc “Additionally, enabling Spectator Mode means that you can observe (or, in certain scenarios, interact) with the VR experience while a player uses a HMD in a VR environment.” It would be nice to know these “certain scenarios” !
Actually I only get success in adding some keyboard interaction (without UMG), but it’s pretty restricted.
I’ve seen various threads on here and AnswerHub with people trying to figure out how to create interactive UMG widgets on the Spectator Camera, but no official response to any of them. The spectator screen feels half-baked at the moment.
Would be great if someone from Epic could address this.
Bumping this. Right now a workaround would be to get the mouse cursor position and have each of your “buttons” have events that fire when the mouse position is in a certain 2d vector range. It wouldn’t be true interaction but it would be enough of a workaround till Epic puts something native in. You could create new variable ranges that correspond with certain buttons so it would be pretty easy to modify once you had it setup.
My problem right now is, I don’t know how to render the retainer box (which is supposed to render to a texture) and assign that texture to the spectator screen. I KNOW I’m doing it wrong I’m just looking for some guidance -
Actually Spectator view is rendered through a SceneCapture2D (and a rendertexture assigned as the view) - which can’t use widgets on it’s own. Perhaps using a 3d widget placed in front of the camera ?
I am working on a similar problem at the moment. The first issue is getting the widget to render to a target texture which is displayed using the new Spectator Screen Mode functionality. As you’ve said Teapot, this isn’t accessible in BP. However, Kyr Dunenkoff from People Gotta Play (linked below) has made a BP function library which allows you to render widgets to target, just a SceneCapture2D does. It works pretty well, but I am still working on how to get the mouse to be able to click buttons and enable text input. I’m going to try what has been suggested above. I’ve attached a screenshot which shows how the BP functionality looks.
Hi guys,
I was quiet busy today to find a solution to that problem on my own, because I really need this for a mixed virtual reality installation. But as you all already realized it’s not natively possible to use the mouse on a widget which gets drawn to the spectator screen as render target aka texture.
What I ended up doing was to split the screen in several parts and display the widget in one of them, while doing the changes to the widget all by keyboard commands. So it basically degraded my widget to an info screen, which is okay for now, because I only wanted to visualize the changes made to write them down or save and load them later on.
But if any of you found a solution to this, I would really appreciate to hear it.
I’m going to try this one more time…I really need this to work for a tradeshow, literally for my livelihood. It’s completely retarded this isn’t doable. I don’t want to have to set every nuanced event and variable in my singleplayer VR project to replicate and host on lan, just so I can click some stupid buttons to drive functionality… nor do I feel like learning Ajax/Javascript and messing with Event Handlers on the Unreal Web Server… I’ll report any progress.
