I am hoping that some members of the community have some suggestions to help me get a new render pass implemented. I’m new to Unreal, but have many years of experience as a rendering engineer working on bespoke engines. I’ve been able find a number of resources online which touch on some important points, however I’m finding it very slow to understand how all the pieces holistically actually fit together.
I’m attempting to implement a new render pass which precedes the base pass that does a visibility buffer style render technique and can blend multiple materials together into the GBuffer. At this stage I’m just trying to get a prototype put together for the standard main view (I realize that I will have more work to do in time to support Lumen etc.). I’ve gotten pretty far with my technique - I implemented a new EMeshPass, a new mesh processor, and all the necessary global shaders to generate all the data I need to apply my visibility buffer to the fullscreen passes to blend the various materials together.
The trouble is in the final pass to invoke the material shaders. I really don’t want to make any changes to BasePassPixelShader.usf if I can avoid it. Instead, I’m trying to write a new .usf that is a trampoline into the base pass pixel shader which decodes my visibility buffer on entry into the shader and applies the appropriate weighting to the results of the base pass in FPixelShaderOut. On paper, it all makes sense but this is proving really difficult as the shader code and bindings feel really intertwined with the flow of FBasePassMeshProcessor. In my case, the material application is just a fullscreen pass - so I’m trying to just get something bootstrapped that is more of an “immediate” style rendering execution.
Effectively, what I’m trying to write is:
// lambda pseudo code…
FGraphicsPipelineStateInitializer PSO; // ← set this guy up
SetShaderBindings(); // ?
It feels so straight forward that I should just be able to “extract” my shader bindings from the material and I’m good to go…
I run into problems, however, if I were to attempt to replicate the logic inside FMeshDrawCommand: where if I were to just do something like put a FMeshDrawShaderBindings on the stack and attempt to call GetShaderBindings() I immediately run into problems where there is no render proxy, because again I want to just draw a fullscreen pass.
I’ve also tried to re-implement this by making my pixel shader a FMaterialShader instead of a FMeshMaterialShader, however that doesn’t work either as again some bindings population is tightly coupled to FMeshDrawCommand.
Another attempt was to pattern match how the post process materials work, but because I am using the BasePassPixelShader I run into problems where I don’t have a vertex factory and I can’t compile my shaders.
My technique is very much like how Nanite does it’s GBuffer population so I took a look in that code, but I got a little scared away there as it is a huge system and I’m just trying to learn how to appropriately setup my shader bindings.
Again, I’m still struggling to see all the connections in the render pipeline so that I can chart an appropriate course, and I’m hoping somebody here might have a suggestion for how to proceed. Some high level questions:
Should my pixel shader be a FMeshMaterialShader? Does this mean that I need to go full Nanite and pre-build draw commands and somehow invent render proxies? Again, this felt super overkill to just draw a pass…
Should my pixel shader be a FMaterialShader? If so, how do I appropriately get shader bindings? How do I resolve the issue with the vertex factory?
Is there a simpler example than Nanite that I can pattern match against which kind of does what I want: use a fullscreen pass with the base pass pixel shader?