Nvidia DLSS & RTX working with VR

So after some testing this seems to work but it will either add to or reduce latency a bit depending on settings. Once plugin is installed and activated just add an event on begin play that sets DLSS mode to balanced or performance in level blueprint. It only manages between 0.5ms and 1.5ms improvement in one project but in others it made things worse…

I also set project to DX12 which seems to give a boost though again depends on project. I would also recommend turning off GPU scheduling in windows and any game overlays plus install latest drivers.

I don’t think it works as well as normal games because you’d expect much more of a boost really. It works great in the viewport and gives massive gains there but not much for VR right now.

Perhaps it hasn’t been optimized yet by nvidia? Hopefully others more knowledgable can make sense of it?

2 Likes

Hey, I don’t have anything to add, couldn’t test it myself yet. But I was very curious about DLSS+VR and just wanted to say thank you for sharing your test with us!

While Nvidia providing DLSS 2.0 to all who use Unreal engine is awesome news, it’s too bad that Epic aren’t working on anything for VR that we know of presently. VR in UE4.26 is too broken presently to justify developing for VR on it (though not impossible, just way more work)…and Epic has said they are only fixing the most important issues in UE4 due to the overwhelming amount of them. Unless UE5 brings a new level of VR capability to the table, it’s probably best to stick with UE4.25.4 for VR until then.

Thanks for letting us all know how this is going though.

Yeah i won’t be using it for now as it’s too early to say if it’s worthwhile for vr. It seems to make a well running project worse but can improve ones that are demanding. I suspect all flat games will want to include it as the gains are massive and in the viewport at 4k it’s quite useful to keep things running smoothly.

What is wrong with VR in 4.26 (as compared to 4.25, not in an absolute sense)?

While most VR features work well in 4.25.4 with a few exceptions like ‘Instanced Stereo’ which is hit or miss depending on shader types in 4.25.4…the following thread will indicate more of what isn’t working properly in 4.26 based on some user feedback.

https://forums.unrealengine.com/development-discussion/vr-ar-development/1664139-niagara-render-broken-in-vr-with-instanced-stereo-on

Hey there, my I ask how you did manage to get this fired up? I use 4.27.2 with an HTC Vive Pro and SteamVR. I used the VR Template as my starting point but as soon as I set the RHI to DirectX 12 and VR Preview crashes Unreal Editor.

Any help would be highly appreciated.

Try switching to the SteamVR beta branch. The current stable 1.20 version will crash UE4 when DirectX 12 is used.

1 Like

WOW! That fixed it. Just starting out with VR in Unreal.

Thanks a lot!

my update and notes:
I am using 4.27.2 for VR dev. So far no crashes. I will also test the new DLSS 2.0.

A few points, I can’t seem to find an answer, any help is highly appreciated.

1. DX12 and Raytracing
For some reason, when I launch the OpenXR Template, and I switch to DX12 and Raytracing, my shadows are not dynamic. When I use a dynamic light source, and I increase for example the size of a spherical light, the shadows stay sharp and hard.

I can’t find the changes or settings inside the template, that’s causing this. The reason I want to use Raytracing for VR, is basically to use the GPU Lightmass for Baking. After baking I’ll package with DX12 or DX11, but of course without Raytracing.

Please let me know your thoughts on this process and all your experience.

2. Quixel Megascan Materials
When using the OpenXR Template, the Quixel Parent Materials are not working.
I am not sure what Shader Model is used, but it would be amazing to find a workaround and to use all the Quixel Assets for the OpenXR template.

Thanks for all your info, hints, tips and experience.
Appreciate it and happy pixeling!

Has anyone done some more testing with DLSS and perhaps Unreal 5.2 Lumen in VR?

I read on Nvidia’s forum that maybe there’s some long standing issues which haven’t been fixed yet (on Nvidia’s part).

EDIT: Seems the DLSS plugin hasn’t been updated for 5.3 yet, which hopefully means that it will include DLSS 3.5 support which apparently has a big impact for raytracing!