My height map is a RAW (Photoshop) file, 16-bit greyscale.
I tried saving it in PNG and changed nothing about the position of the 0 :t
So, what you mean with the shift?
here I exaggerated the scale to show it better, but the problem is there from already z=5