Voxel density does decrease when you move away from objects. And there is no control over the rate of decrease, other than the overall range parameter.
You can try using the new with clip-map placement control (see comment), and place the anchor near the reflected object.
Also you can use SSR or set “r.VXGI.CombineSpecularWithSSR 1” to get high-quality reflections of visible objects.