Scene depth not consistent across view

If my background is a sphere centered on the camera then scene depth should be consistent over that sphere should it not? As every face of the sphere is equal distance from the camera.

What I have found is I get almost a vignette effect, where I am getting inaccurate depth values towards the edges of the view. This is ruining an effect I am trying to achieve. Is there an explanation for this and a way to fix it?

I have attaced a picture as an example, the top half is what it looks like, the bottom half I have increased the contrast to highlight how it is darker around the edges than the middle.