How To Hide Post Process Outlines When Object Is Occluded

It’s a max of the custom stencil mask (R) and the neighbouring pixel. Look in the shader.
It has some extra clamps to limit it to the 0-255 scale of the custom buffer.