Hey, I’m playing around with GBuffers/Custom shading models/BasePass stuff etc.
Obviously there are little to no sources regarding these topics and me being not that comfortable with rendering side of things I’m finally out of options so I’m trying my luck here.
I need to mask out materials that are using Translucent blending mode (after DoF) with custom values, but without using custom depth/buffers.
I somehow managed to make it work to the point where I have the data I need, but unfortunately I have no idea where to look next to resolve this issue.
This is regular working transparency where I have two image planes at different depths and alpha blending works as expected → I can see the image through transparent parts of the foreground image.
With my approach I’ve been able to get here:
The red part is the image plane further away and in front of it is the one that should’ve been transparent. I’m getting the right values, but instead of blending the “black” parts of the image with the red background, the color gets written on top of it.
By looking at alpha of separate translucency buffer I’d like to get something like this
but color coded - e.g. red/white.
I’ve found out that in RenderTranslucency function of FDeferredShadingSceneRenderer the output in scene color comes out already composed the right way when leaving the call to RenderTranslucencyInner for TPT_TranslucencyAfterDOF pass. I’ve binded my own render targets in RenderTranslucencyViewInner like this
and in BasePassPixelShader.usf I’m referencing MRT[2] like this
All of this was done by trial and error and I’ve came upon these issues/things:
-
MRT[1] is bound to velocity buffer and I have no idea where it is coming from. That’s why I cannot bind only one new
FRenderTargetBindingto slot 1, because it gets overwritten by this awesome piece of code
-
No idea what RETURN_COLOR does - some comment about type promotion/coercion
but since I’m binding to slot 1 and 2, I’m not really sure if I’m losing velocity info, but atm. I don’t need it and I haven’t found anything anywhere about the velocity binding to MRT. I know it is enabled in ShaderGenerationUtil.cpp like this (with my own addition):
But I haven’t found any actual buffer binding and methods for GBuffer generation are not being called during transparency rendering so this is all a little bit unclear.
However, I’ve played around with chaging formats and bound indices for my custom RT and original SceneColor and I found out that only texture, no matter if mine or original, bound to slot 0 is getting the composition right. I suspect that somewhere is said to blend only RT bound to index 0 but I am not able to find the place where such thing is being specified.
I am able to get parts of background image by clipping/discarding the value in base pixel shader, but that is effectively just Masked material and I’m losing my smooth transition around the edges.
I am not even able to find where it is said the pipeline should use BasePassPixelShader for this, I just know by changing it that it works, but now I’m seriously lost.
If anyone has any ideas what could I possibly do next, I would very much appreciate it. I believe somewhere there is a setting that says to blend by using depth stencil for RT bound to slot 1, but it is well hidden/implicitly set.
By going through code I ended up in DoTask method of FDrawVisibleMeshCommandsAnyThreadTask class which is used for drawing batched meshes in parallel, but it’s not really telling me anything useful.
I’m not using strata of OIT, just basic translucency with SM5 and post DoF. I understand there are probably 5 people who could help me with this out there, but I’m completely out of options.
I’m not really sure about these concepts, but it seems so.