Put 3 scene texture samples in your material, offset the UVs used to sample each of them slightly differently based on the surface normal which can be retrieved with another scene texture sample. Take the R value of one, the G of the next and the B of the 3rd then append them back into a single RGB vector.