I’ve actually been able to use it on Skeletal Meshes in the end, and it’s working!
First, the “parasite” deformations were caused by not checking “Use full precision UVs” on the Skeletal mesh.
With that in mind, to create a meaningful offset for a skeletal animation, instead of generating the textures from the deforming model itself, you can make it from a difference mesh between the Skeletal+Blendshapes and Skeletal only (in Houdini).
I applied it on a Skeletal Mesh and seems to work nicely. The only problem is to match the speed of the Skeletal animation with that of Vertex Animation.
Still, until Houdini gets the ability to export Blendshapes in FBX, this can be a workaround.