Mixed Reality Status update

Can this be done without recompiling the engine an dusing the release from Epic?

(I mean using VIVE trackers)

Cheers.

Not right now. The recompiling adds in the ability to render the quad view from texture (i think). You can use a 3rd controller and trackers, but you won’t get the quad view output.

The 4.16 commits on github mention social screen support (running a different view on the TV) for PSVR. Haven’t checked yet to see if it included anything for running a second view on the mirror viewport on Rift/Vive, but seems like it might.

Yes, i downloaded the 4.15 and also 4.14 source code, run the setup and projectfiles bat and tried to compile everything in Visual Studio. Without the addon everything works fine.
Maybe “E:\GitLab\UnrealEngine-4.14\Engine\Plugins.…” is your datapath and its not updating to my path (“E:\Mixed Reality\UE 4-14\UnrealEngine-4.14.3-release\Engine\Plugins.…”)?
Im not sure how to fix this. I also tried to put everything in the same location (E:\GitLab\UnrealEngine-4.14\Engine\Plugins.…).
The only other thing i did was to download the sourcecode for myself and not with github.

What you mean with “install the c++ stuff”? Do i need some libaries for that, that i dont need with the normal source code?

Thanks for your time.

As long as you copy the mixed reality files/folders in the correct place (it should overwrite some files) then that part should be good. I had to “install c++ stuff” because i guess my visual studio did install the necessary things to compile the c++ libraries. So I think I had to create a new c++ project in visual studio, then it asked if I wanted to install the c++ stuff

I haven’t actually tried those demos firsthand yet, though I was super excited when they were released last year. I’m definitely not saying it’ll be a bad experience to try to hold an object that doesn’t exist, just from a cognitive perspective (which is my area) there would be some misalignment in sensory…

I theory it’s possible to use Slate to create a new window and render to it. A freind of mine did this for a different purpose. I’ll continue working on this MR-Stuff soon (next week?) and I’ll try to get s solution up and running that doesn’t require a custom engine version. For those interessted in my material, copy the text here: Google-Drive and just paste it in the ForwardPost-Material and should create all the nodes :wink:

Greetings

I´m really interested, but I would like a bit of an explanation before I try your nodes because I have a deadline near, so I´ll try to wait, but I will need to try to do it in a week or so :slight_smile:

Cheers and thanks!!!

@tweaq can you provide your Steam plug-in folder that compiled? I have tried it on 4.14, 4.15 and 4.16 with no luck. I noticed all these builds have been updated within the past four months or sooner. The Mixed Reality plug-in, six months ago. I also, save the original Steam folder and placed it back into the 4.14 build and after doing a Build “clean” I was able to rebuild correctly.

Any luck with this? I will delve deeper if you know this to be true. I have deadline this Friday for my show “3D in Review” and want to use some mixed reality function and sadly the compiling even in 4.14 did not work for me.

4.16 preview is here and no mention about Mixed Reality… Zak Parrish was just live in the Unreal Engine Channel and when asked about it said that he “couldn´t tell when its coming…” That´s just said…

I don’t know if you read my other posts to this thread, but I cannot get it to compile correctly, a number of C++ errors in the SteamVR plug-in cpp and .h files that are not there with the existing build of 4.14.x from GitHub. I have been creating builds from Unreal for a few years and the problem you mentioned is due to Visual Studio not installing C++ by default, hence when you created a C++ project it gave you option to download the c++ libraries. This would have happened even with a good build. Anyway, what version of Visual Studio are you using and build of 4.14.x are you using? Plus, can you just make the new plug-in (folder) available? I am pretty sure there are no dependencies so the properly compiled plug-in should work in the place of the current SteamVR plug-in.

I hope you see this and can help.

Thanks!

manual

I got it running in 4.15 after i copy all the code for my own (marked in code with //Mixed Reality) into my running engine code. Before this i got running an early implementation from here: GitHub - haowang1013/UE4-MixeReality: Some stuff for doing mixed reality recording in UE4
Its allmost the same code like this from the post here: https://github.com/richmondx/UE4- only that richmondx add some modification to get some forground calculations.

You only(!) need to copy the code from SteamVRFunctionLibrary.h (…\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Classes)
and SteamVRFunctionLibrary.cpp, SteamVRHMD.h and SteamVRRender.cpp (…\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private)
and recompile everything completely new(!).

If you got some problems with this line (you will get problems):


RHICmdList.Clear(true, FLinearColor::Black, false, 0, false, 0, FIntRect());

you have to replace it with this line:


RHICmdList.ClearColorTexture(BackBuffer, FLinearColor::Black, FIntRect());

After compiling and seting up you scene with the blueprints (like BP_VRCapture2D)(Or first try the downloaded MR Sample), you have to set this:


[SteamVR.Settings]
WindowMirrorMode=3

in the Engine.ini file of your project.

Some optional stuff:
With the code from tweaq, you put it after “else if (WindowMirrorMode == 3){:” in the SteamVRRender.cpp,



RHICmdList.ClearColorTexture(BackBuffer, FLinearColor::Black, FIntRect());
			
			RendererModule->DrawRectangle(
				RHICmdList,
				0, ViewportHeight / 2,
				ViewportWidth / 2, ViewportHeight / 2,
				0.0f, 0.3f,
				0.4f, 0.4f,
				FIntPoint(ViewportWidth, ViewportHeight),
				FIntPoint(1, 1),
				*VertexShader,
				EDRF_Default);


you can also add the view from the vr headmount.
Also you need to remove RHICmdList.ClearColorTexture(BackBuffer, FLinearColor::Black, FIntRect()); in a later codeline.

And with the blueprint action from you got some other fancy stuff.

cheers :cool:

Here is a list of error codes I received.

Severity Code Description Project File Line Suppression State
Error Expected SteamVRChaperoneComponent.h to be first header included. UE4 D:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRChaperoneComponent.cpp 1
Error Expected SteamVRFunctionLibrary.h to be first header included. UE4 D:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRFunctionLibrary.cpp 1
Error Expected SteamVRHMD.h to be first header included. UE4 D:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.cpp 1
Error Expected SteamVRSplash.h to be first header included. UE4 D:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRSplash.cpp 1
Error (active) identifier “FLayerDesc” is undefined UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 139
Error (active) identifier “FLayerDesc” is undefined UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 141
Error (active) identifier “FLayerDesc” is undefined UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 142
Error (active) incomplete type is not allowed UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 34
Error (active) incomplete type is not allowed UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 420
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 139
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 140
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 141
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 142
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 143
Error (active) member function declared with ‘override’ does not override a base class member UE4 d:\GitHub\UnrealEngine-\UnrealEngine\Engine\Plugins\Runtime\Steam\SteamVR\Source\SteamVR\Private\SteamVRHMD.h 144
Error MSB3073 The command “…..\Build\BatchFiles\Rebuild.bat UE4Editor Win64 Development -waitmutex” exited with code -1. UE4 C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\V140\Microsoft.MakeFile.Targets 46

This was done is Visual Studio 2015 and as mentioned, swapping the original folder back in, builds properly and yes in those .cpp files the headers are in the proper order.

Works greatly :slight_smile:

Do anyone have a idea why i got everything running in the sample project, but after i put everything in my project, the two rendered MR views are both the same?
I mean normaly one view (upper left) is green with the “Render CustomDepth Pass = true” marked objects included, but for me i see two times the same view with all actors (normaly only bottom right).

Cheers and thanks in advance.

You need to copy everything for your own like i descriped here: Mixed Reality Status update - XR Development - Epic Developer Community Forums
Use a clean engine code build.

Thanks, but the instructions are erratic from those in the GitHub code and what you are asking me to do. I have worked with a clean build as well, on several attempts now. I will try again with your instructions, and hope for the best. One more question, what version of Visual Studio are you using to build this version?

Im using Visual Studio Community 2015.

I wish you success.

Sadly, even with your instructions it will not compile correctly. It is frustrating that you don’t find this out until the you get to the section for plug-ins, which is about 45 minutes in, but I still get the same error, which is the includes are not in the right order:

#include “SteamVRPrivatePCH.h”
#include “Classes/SteamVRFunctionLibrary.h”
#include “SteamVRHMD.h”

I am using the latest GitHub build for 4.15.1 downloaded through the GitHub desktop app for Windows, replaced the four files you specified:

SteamVRFunctionLibrary.cpp
SteamVRFunctionLibrary.h
SteamVRHMD.h
SteamVRRender.cpp

I then ran the Setup.bat (Admin Rights), GenerateProjectFiles.bat (Admin Rights) and finally double clicked on UE4.sln which launched Visual Studio 2015, where I selected Build UE4. This is how I have compiled several UE4 builds over the past couple of years.

As mentioned, if you compiled this with the latest 4.15.1, then your binaries and SteamVR.uplugin should work without having to compile on my own. At least this is how I have been able to add plug-ins in the past.

Adding back the original four source code files resulted in a successful build.

4> Total build time: 1707.54 seconds (Local executor: 1604.99 seconds)
========== Rebuild All: 4 succeeded, 0 failed, 0 skipped ==========