Detail lighting not rendering correctly in Raytracing with HDRI.

Included a video to demonstrate the issue.

The detail lighting pass looks like a muted version of the lit pass, it retains all the diffuse color and texturing.
As soon as I delete the HDRI, and put a light in the scene, the detail lighting pass is normal again, so the HDRI (or HDRI when used with RT) is the culprit, but what’s the solution?

A little extra detail on why this matters, I need the render elements for compositing in Nuke. I’m trying to get shadow mattes ideally.

Bump, still haven’t solved this issue.