Thanks for the info.
I think getting correct silhouettes and fixing POM on curved surfaces go hand in hand. This is what it currently looks like with POM on a curved surface:
The positions at the surface of the mesh are correct but the shader assumes that the height plane within each triangle is parallel to the surface polygon which is not the case when there is curvature. So it pulls the underlying surface along the curvature instead of actually curving.
It is almost like a lens effect when you spin around an object like this.
The root cause is stepping through the heightmap in tangent space. It may be possible to step in world space but it will cost more inside of the main loop. The other idea I have is to try to get the curvature of the surface at the point of entry (perhaps ddx,ddy on vertex normal) and then determine how much Z offset the ray needs to get at each step. So it has a chance to escape by bouncing off the plane. That may be cheaper but require more difficult math.