How do I do this masking effect?

Seriously, what?

Are you talking aboiut Custom Depth Stencil?