Preview Window | Independent resolution and UI?

Hey peeps,

I want to discuss something with you today.

As some of you might know the Preview Window of the Vive (the window you have on your PC while playing a VR game on the Vive) shows a part of the left eye.

Currently “WindowMirrorMode 1” gives us a vertical image. Based on information from the forum I created a PullRequest to at least give us a horizontal resolution. But this is still not really optimal.
The Vive has a resolution of 2160x1200. Means one eye is 1080x1200. The SteamVRRenderer maps the left eye to a range of 0 to 0.5 (horizontal) and and the right one to a range of 0.5 to 1 (also horizontal).
Both are mapped to 0 to 1 vertically. It also applies a black texture to it to surround the eyes (why ever…).

This results in the image which was posted on the forums. I can’t find the Thread/Author at this moment, so who ever did that, thanks again!

[spoiler]

&d=1469637397

[/spoiler]

The blue rectangle is the current “WindowMirrorMode”. The green one is the one the PR created (although my PR shifts the green one .5 to the right, so it’s in the middle of the eye).

While this create a horizontal rectangle it’s still not showing all of what the actual player sees. Now my question is: Why are we even taking the eyes for the Mirror Window?
Why can’t we query what the Camera, which the HMD is bound to, sees? That should have the normal (for example FullHD) resolution.

####################

The second question or discussion point is: How can we add Vive-independent UI to this window?

Games like “AudioShield” can be controlled via this Window while the VR Player is not getting affected by it.
This allows people to change things without interfering with the Player. It would also allow us to display information on this screen, like score etc.

AudioShield is created in Unity and I don’t have an example for UE4. That makes me feel like UE4 doesn’t support this out of the box.
I’m sadly not skilled enough in Render Thread things to know what I would need to do to enable us to actually have a WindowMirror, that shows a FullHD image, not bound to the Vive Resolution which can also have its own UI (Widgets).

####################

So if anyone here is interested in this, or maybe even knows a good chunk of the solution, it would be more than awesome to share and discuss our knowledge here.

My idea was to change the “SteamVRRender.cpp” a bit more by actually changing this:

[spoiler]



void FSteamVRHMD::D3D11Bridge::FinishRendering()
{
    vr::VRTextureBounds_t LeftBounds;
    LeftBounds.uMin = 0.0f;
    LeftBounds.uMax = 0.5f;
    LeftBounds.vMin = 0.0f;
    LeftBounds.vMax = 1.0f;

    vr::Texture_t Texture;
    Texture.handle = RenderTargetTexture;
    Texture.eType = vr::API_DirectX;
    Texture.eColorSpace = vr::ColorSpace_Auto;
    vr::EVRCompositorError Error = Plugin->VRCompositor->Submit(vr::Eye_Left, &Texture, &LeftBounds);

    vr::VRTextureBounds_t RightBounds;
    RightBounds.uMin = 0.5f;
    RightBounds.uMax = 1.0f;
    RightBounds.vMin = 0.0f;
    RightBounds.vMax = 1.0f;

    Texture.handle = RenderTargetTexture;
    Error = Plugin->VRCompositor->Submit(vr::Eye_Right, &Texture, &RightBounds);
    if (Error != vr::VRCompositorError_None)
    {
        UE_LOG(LogHMD, Log, TEXT("Warning:  SteamVR Compositor had an error on present (%d)"), (int32)Error);
    }
}


[/spoiler]

To only display the left eye from 0 to 1 horizontal and then map the green rectangle to the whole screen, but this would in the end only result in a 1080x1200 resolution. Not really ideal.

So what are your thoughts on this? Has this already been solved maybe? :x

Well for one, because that would require rendering the scene a 3rd time, obliterating whatever remnants of a framerate the Vive had left us with in the first place.

It would be good to control, makes me think of a multiplayer VR game i saw where the PC player (on a laptop) was seeing through the “eyes” of the Vive player, but in regular HD, and with all the enemies visible and the vive player on their PC and headset was listening to commands from the laptop player about where to shoot.

This could be a potential solution - Multiplayer connection, 2nd PC renders the full HD from a regular cam pawn attached to the vive player cam?

Otherwise, actually controlling the resolution of that window and rendering a controllable camera with view port size options, and maybe even custom post process options on that camera, would be beautiful!

I see, I might have used the wrong words then. Would it be possible to just take the left eye of the Vive, cut away the black, round areas and making sure it’s 1920x1080?
So essentially not re-rendering it, but extending what is already rendered to hit FullHD? We could also upscale it to fit the resolution, or?

The Lab also has a custom WindowMirror, also done in Unity. Maybe I need to dive into Unity a bit to see what they go. Sadly they do not share the Source…

It doesn’t support UI for desktop view out of the box, afaik. I tried tossing up some widgets and hacking my way around to see if I could get it to work, but no luck.

It’d be a lot easier for recording gameplay (or for making YT or twitch content) if the desktop view was more like the suggested green box. I know of a few YT’rs and streamers who are very hesitant to create content with UE4 based games because of verticle box. There was a post on reddit a few months ago that outlined a method for creating a full-screen desktop view, not sure if that is the same one listed here.

I’m struggling with this as well.
I followed this tip (UE4 + HTC Vive Fullscreen VR Tutorial - YouTube) to get the desktop preview to be 16x9, but it still starts fullscreen, and if I alt-enter to go windowed, the window is huge, and bleeds off screen.
I’d REALLY like to be able to render that cropped image in a window at that native res, or at least have a way I can provide the window size so I can tweak it myself.
Is this possible? I’ve tried playing with console commands and command line args, but haven’t had any luck. r.setres 1280x720 gives me a 1280x720 window like I want, but breaks VR completely :frowning:

Thanks!

UPDATE:

I found I could achieve what I wanted by editting line 1026 of UE4/Plugins/Runtime/Steam/SteamVR/Source/SteamVR/Private/SteamVRHMD.cpp to read:
FSystemResolution::RequestResolutionChange(1600, 900, EWindowMode::Windowed);

Yeah that’s what my PR is doing.

Will have a look at these lines. There needs to be a way!

Pushing this into the new year to get some more attention to it (: