I seem to have solved it:
To solve it I do this: for any neighbour pixel I take the difference between the custom depth of the current pixel and the custom depth of the neighbour pixel; if the result is greater or equal to a given threshold then I don’t draw the outline, otherwise I draw it.
Then I check if the result neighbour pixel has the same stencil value of the current pixel; if so, I ignore the result, otherwise it will completely fill the object instead of drawing the outline.
The threshold musut be a low value. I’m using something like 0.001, but it still has some problems when objects with different stencils overlap, like this:
I have no idea how to fix this.
Also, if different objects have the same stencil it doesn’t draw the outline, like this:
I would like that different objects, even though they might have the same stencil, had an outline around each one. But I don’t know how to do that either… how could I distinguish between differet objects knowing only the depth information?
It’s ok right know, but I would like to fix these problems… I’m out of ideas.