I am currently on the same path as you using World Position Offset in unreal 5, accessing the height map texture CPU side is going to be very expensive. If possible, you might be able to cache the texture into a TArray and use that to access the height map data more quickly than a texture. The problem is when you need to go between Texture2D and TArray which is where you’ll see slow down again.
I am thinking of making a system that uses a compute shader to either generate the collision mesh at runtime, or output the height data in a more digestible format on the CPU end. Did you manage to get anywhere on your solution? I am still trying to figure out where to start and what method might be best.