I created a pull request that takes both the visible and the hidden area meshes from SteamVR. It isn’t that big a performance change though - the visible area mesh isn’t used that much for postprocessing. That is a separate issue - it probably should be used more.
SteamVR visible mesh bugfix
Better to raise a bug in OpenVR to try get Valve to pass through the hidden/visible meshes from Oculus (and other) runtimes than to hardcode it again, at least longer term. If you need to handle both through SteamVR now, easy to add locally.
With the post process side of things, Epic have started to make things more configurable (global shaders), but can’t really tweak to be ideal for VR yet in a generic way.
“Responsive” when set in stencil fixes the blend to 50% IIRC - it doesn’t completely disable TAA. The TAA history already handles the masked regions fine - you don’t notice when the history hits areas that were masked as they’re low in terms of visible impact anyway. Avoiding running the TAA shader on the masked areas may improve performance. After that, it’s likely radial denisty masking that may offer more consistent gains. But again, right now awkward to implement in a way that is likely to be accepted to trunk by Epic due to maintainability. Hopefully the pipe will become more configurable, then such things could go into plugins. If you haven’t already, it’s worth grabbing a RenderDoc capture to see exactly when the hidden/visible meshes are being used and what effect they have.
Grabbing a frame with RenderDoc is the ideal way to verify. are you going to make a pull request?
I might make one, because TAA is a big part of the post processing cost for lots of projects even though a lot of stuff is moving to forward rendering. They only recently changed the code, before 4.16 it passed Context.HasHMDMesh() for TemporalAA. I’m trying to figure out if there is any plausible scenario where it would matter.
The engine uses a hidden area mesh and a visible area mesh for VR to save on rendering time. A major regression is causing the engine to ignore the visible area mesh, resulting in a ~15% increased cost in post processing. This affects all engine versions, because it is due to the API returning a new hidden area mesh.
The engine asks for the hidden area mesh and then performs a CRC on it. If the CRC matches what is returned from the API, it then uses a hardcoded visible area mesh. The CRC of the hidden area mesh from the API is no longer matching the hardcoded CRC, resulting in the hardcoded visible area mesh no longer being applied.
SteamVRHMD.cpp, FSteamVRHMD::SetupOcclusionMeshes:
if (HiddenAreaMeshCrc == ViveHiddenAreaMeshCrc) // <--- NO LONGER MATCHES
{
VisibleAreaMeshes[0].BuildMesh(Vive_LeftEyeVisibleAreaPositions, VisibleAreaVertexCount, FHMDViewMesh::MT_VisibleArea);
VisibleAreaMeshes[1].BuildMesh(Vive_RightEyeVisibleAreaPositions, VisibleAreaVertexCount, FHMDViewMesh::MT_VisibleArea);
}
This might have been in place for historical reasons, but since OpenVR 1.0.4 (~November 2016) there has been an API to get the visible area mesh directly:
virtual HiddenAreaMesh_t GetHiddenAreaMesh( EVREye eEye, EHiddenAreaMeshType type = k_eHiddenAreaMesh_Standard ) = 0;
//GetHiddenAreaMesh(EVREye::Eye_Left, k_eHiddenAreaMesh_Inverse);
//GetHiddenAreaMesh(EVREye::Eye_Right, k_eHiddenAreaMesh_Inverse);
//^---- call for left and right eyes with the k_eHiddenAreaMesh_Inverse parameter instead
// of the default standardGreat, thanks for that, I was starting in on one too.
When I found this I was originally checking if Oculus got a hidden area mask when using SteamVR: it doesn’t =(.
I think it could be worth adding the same hardcoded mesh that the Oculus plugin uses into the steamvr plugin if and only if the headset is the rift and steamvr returns no meshes (it currently doesn’t). So it would only act as a fallback.
Otherwise, as it is, the Oculus plugin is currently getting an artificial performance advantage against the SteamVR plugin when used with the Rift.
(I looked through a bit to see if it was used in most passes. All of these call DrawPostProcessPass and pass in “Context.HasHmdMesh()”:
Private/CompositionLighting/PostProcessAmbient.cpp
Private/CompositionLighting/PostProcessAmbientOcclusion.cpp
Private/CompositionLighting/PostProcessLpvIndirect.cpp
Private/CompositionLighting/PostProcessPassThrough.cpp
Private/PostProcess/PostProcessAA.cpp
Private/PostProcess/PostProcessBokehDOF.cpp
Private/PostProcess/PostProcessCircleDOF.cpp
Private/PostProcess/PostProcessDOF.cpp
Private/PostProcess/PostProcessEyeAdaptation.cpp
Private/PostProcess/PostProcessHierarchical.cpp
Private/PostProcess/PostProcessMaterial.cpp
Private/PostProcess/PostProcessMotionBlur.cpp
Private/PostProcess/PostProcessNoiseBlur.cpp
Private/PostProcess/PostProcessSubsurface.cpp
Private/PostProcess/PostProcessTonemap.cpp
Private/PostProcess/ScreenSpaceReflections.cpp
As long as “Context.HasHmdMesh()” is in the context, they should render through the visible mesh instead of to a full quad.
Only these seem to not use it:
PostProcessBloomSetup.cpp
PostProcessBokehDOFRecombine.cpp
PostProcessDownsample.cpp
Private/PostProcess/PostProcessTemporalAA.cpp
)
One other note: internally, Temporal AA has a feature for translucent materials with “Responsive TAA” set that has them write to a stencil buffer that tells the TAA shader to skip TAA processing and weight the current frame in that region at 100%. Maybe it could be possible to tap into that system in order to make this work for TAA. I think it would need some kind of reprojection though, because we want to say areas that were masked out in the previous frame should be weighted 100% with current data.
Yeah, the problem I was thinking of wasn’t that it might draw data into the masked regions that you can’t see anyway, it was that it might pull in black from an area that was masked in the previous frame but isn’t anymore (since it uses velocity etc. to find where to pull from), causing smearing.
However, thinking about it more, the way it does things with a quad right now would have the same problem, so the neighborhood filtering or something must already be minimizing that effect anyway.
I’ll try and mess around and see what happens when I run it over the visible area mesh instead of the full quad.
I tried it out and it seems to work fine. I changed all calls to DrawPostProcessPass in PostProcessTemporalAA.cpp to pass in
Context.HasHmdMesh(),
instead of
//false, // Disabled for correctness
I also verified in the debugger that it was executing them (it may not if using the compute shader path).
I also turned on the full mirror to see if there was any different black smear or anything being pulled in and I couldn’t see any when moving it around wildly.