Using Custom Depth as a mask - how to get antialiasing?

I’m trying to create a primitive render layer solution for Matinee, where I can render out individual assets with a corresponding alpha.

I’m currently hijacking the ‘use custom depth as mask’ feature from HighResScreenshot, which does give me an alpha, but it is a purely binary image with no antialiased greyscale values. So the RGB values are antialiased but not the alpha, making it not terribly useful for compositing/postprocessing. Presumably the masking pass happens after the antialiasing is done?

Does anyone have any thoughts on how to get an antialiased alpha?

More generally, are there engine guys at Epic that would be guide us through some of the rendering code?