I have a slightly better way of working around that these days but it will cost. Basically you need to bake out the full transform of the terrain into a texture. I do this by storing the Tangent X vector and the Vertex normal. In order to store both of them, I convert them to octahedrons and write it so that RG = Tangent X and BA = Vertex normal. Octahedrons are used since the values must be worldspace and the octahedron is cheaper than trying to derive Z since the Z sign would have to be bit packed somewhere which also hurts precision.
Then the transform texture must be hooked up alongside the heightmap to the POM node. Then the POM code transforms the camera vector into the new tangent space once per iteration instead of just at the beginning or starting position. This allows the ray to be corrected as it moves through the curved tangent space. This is actually the standard way of handling curvature for cases where you aren’t slicing the mesh into prisms and handling it more using a geometry shader approach, but that method is far more complex and costly from what I gather.
I will try to post how I did that and some new code soon. If I do not get to it feel free to bump the thread or PM me since all of the work is done but I am jumping between lots of tasks right now so it is easy for me to forget about these things between workdays.