So with stencil layers, I can do a bit mask to mask out only one stencil layer. However, if I have 2 stencil layers, and want to compare the depth of each layer to one another, how can I go about that?
Edit: okay actually I should probably phrase this better. So if I have two stencils on different stencil layers, when they overlap it (I assume based off this image) adds the bits together.
I don’t think you can, at least not without engine modification. The depth pass a full 0-255 bit range. A mask only uses a single bit per mask, which is why you can have so many of them.
With engine modification, you could theoretically have as many depth passes as you want, but there are a variety of reasons why this probably isn’t a good idea. There is not unlimited memory to hold all these depth buffers, nor is it free to keep drawing them.
I would think outside the box if there are other more efficient ways to achieve what you’re trying to do that doesn’t require rendering 3 or more depth passes.
For example maybe you could create a 3D distance field texture that represents your object and could ray march in the post process to determine the depth of the object, and compare it to others.