Scene color to get only objects behind the material

I’m using Scene Color channels to achieve a chromatic aberration effect on a translucent surface, but what I found is that every object that is before the surface gets rendered too.

Is there any way to avoid this problem?

[video]https://imgur.com/a/8ISOtb8[/video]

Material code used:

Surface
Translucent