I know it is an old question but I wanted to share a way to achieve that effect since I couldn’t find one anywhere.
Just modify M_Highlight in the following way:
To outline only unoccluded parts of objects:
To outline only occluded parts of an object just switch the inputs of the if node (A>=B and A).
What it dose is the following:
When we want to draw an outline we compare the depth value of the custom depth buffer and the depth value of the buffer at that uv.
If the value of the depth buffer is less then the value of the custom depth buffer at that uv then it means that that uv is occluded and we discard the outline draw at that uv coordinate.