I reproduced this bug by lowering the quality settings.
Best way to see this effect, is use epic settings… then switch to low or medium (Quick settings → Engine Scalability Settings).
The heightmap is rendered in the GPU, its logical that when you lower the quality (and thus lowering the quality of texture mipmaps etc) this will affect the way the heightmap will be rendered (bigger inconsistencies when calculating the different lod-zones in the gpu shader).
Either force this texture to be on epic quality somehow (don’t think there is a manual override per texture currently), or tweak the gpu heightmap shader.
Also a dev responded on this issue, its probably going to be fixed into 4.5 but not sure: