I am currently working on my master thesis which is about Foveated Rendering in Unreal Engine.
I am at the very beginning and I currently try to implement the method from this Paper as a reference method.
The core idea is to save some performance by creating a probability distribution and sample from that. Then utilizing Early Z-Culling the Shaders will be invoked only on sampled pixels. The rest is then interpolated efficiently and presented to the user.
So far I have managed to create a simple distribution only based on Visual Acuity and a binarized Gradient of the Depth Texture (right before the BasePass). When a sample is generated I add a small Delta to the Depth-Texture which prevents the Basepass from rendering non-sampled pixels:
Right before SceneColor is resolved the pixels are interpolated with an algorithm called ‘Pull-Push-Interpolation’ resulting in the final Scene (FeatureMap is a visualization of the propability distribution used for Sampling):
Now I have the problem that I need to access the WorldNormals for the Silhouette-Part of the distribution.
So is it possible to move WorldNormal-Rendering from BasePass to the PrePass?
I haven’t looked too deep into the rendering code so far and unfortunately my Computergraphics and C++ knowledge is rather basic than advanced. But I take up this challenge so far
Another thing is that I will need Saliency-Textures in the future which should ideally be precomputed offline. Can anyone give me an advise where to start looking for possibilities to realize that? Maybe by modifying the Editor to compute a Saliency-Texture when a Texture is loaded or saved?
Thanks in advance!
I hope everything is clear, otherwise I will try to explain it more clearly.
Best,
Ceron
Edit: I noticed that TextRenderActors seem to not ne present in SceneColor when SceneColor gets interpolated. Any Ideas how to fix this?
Modify second branch in FDepthDrawingPolicyFactory::AddStaticMesh such that PositionOnly buffers will never be used (bad idea, because this is an optimization, but I was unable to change it to a ‘PositionAndNormalOnly’-Buffer so far - any suggestions?)
Move GBuffer-Allocation in FDeferredShadingSceneRenderer::Render right before RenderPrePass() is called
Override bNeedsPixelShader in FDepthDrawingPolicy’s constructor with true to force the factory to use pixel a pixel shader
make FDepthOnlyPS::ShouldCache() always return true to make the shader available in the ShaderMaps
Change BlendState in SetupPrePassView() to TStaticBlendState<CW_RGBA>::GetRHI()
in FSceneRenderTargets::BeginRenderingPrePass() Set ColorTarget to GBufferA->GetRenderTargetItem().TargetableTexture
in DepthOnlyPixelShader.usf and DepthOnlyVertexShader.usf I had to redefine MATERIALBLENDING_SOLID to 0
Finally I take the Normal from MaterialParameters.WorldNormal and do the same as in EncodeNormal() in DeferredShadingCommon.usf: OutColor = float4((Normal0.5+0.5),1.0)*.
I am not sure about the sideeffects of these modifications (particularly relating to performance which is crucial for my thesis, but with no alternatives I will use it this way and compare everything using this modifications).
Just wanted to say that this looks awesome. I’m assuming you’re aiming for improved VR performance with the project? Any idea of what kind of performance savings would be possible?
Thank you! Yes I am aiming for better performance and also not creating any perceivable differences. In the linked paper the authors state they achieved an overall reduction of rendertime by roughly 25% in their non-optimized prototype. These savings depend on the per pixel shading cost. In my own method I’d like to address geometry as well (e.g. by using lower LOD-Levels in the visual periphery but I am not sure yet) to further improve the performance.
Because Unreal Engine is highly optimized already, I think the possible savings will be (much) smaller and I’d have to guess what kind of savings would be possible. I am very curious about how much performance could be saved and try to do my best (obviously ), so let’s see
Just in case anyone else could be helped out with these information: The problem is now kind of solved!
Comment out the following code section in void FDepthDrawingPolicyFactory::AddStaticMesh(FScene Scene, FStaticMesh* StaticMesh)*
if (!Material->MaterialModifiesMeshPosition_RenderThread())
{
// Override with the default material for everything but opaque two sided materials
MaterialRenderProxy = UMaterial::GetDefaultMaterial(MD_Surface)->GetRenderProxy(false);
}
You then need to make sure that the DepthOnly VertexShader will be available in the ShaderMap. So I added the following at the beginning of TDepthOnlyVS::ShouldCache(EShaderPlatform Platform,const FMaterial Material,const FVertexFactoryType* VertexFactoryType)*:
FString FactoryName = VertexFactoryType->GetFName().ToString();
if (FactoryName.Equals(FString(TEXT("FLocalVertexFactory")))) {
return true;
}
In void FStaticMeshSceneProxy:: DrawStaticElements(FStaticPrimitiveDrawInterface PDI)* there are created many instances of FMeshBatch which will be inerted into the StaticMeshDrawLists together with the corresponding DrawingPolicies. For the DepthOnly Pass this is done by use of GetShadowMeshElement() which I now replaced with GetMeshElement().
), so let’s see