I’m encountering some problems playing a vertex animation in Unreal at the correct start point and play speed and was wondering whether anyone could shed some light on what I’m doing wrong.
Basically I followed this tutorial to generate vertex animation textures in 3ds max to import into Unreal Engine 5.2:
The idea is to use a vertex animation texture to animate an NPC’s lips during dialogue. Dialogue is set up in the Behaviour Tree. The lip animations themselves were generated by Audio2Face, imported into 3ds Max, with the .exr and .bmp textures generated using VertexAnimationTools.ms. The vertex animation material on the NPC is set up like it is in the video like so:
And the blueprint task in the Behaviour Tree that handles playing voice acting, displaying subtitles, and lip animation is set up like so:
Nothing too complicated.
In a nutshell, I have a separate material instance for each lip-sync animation, and switch material instances on the NPC’s head to match the current line of dialogue. This part actually works fine believe it or not! However, when I switch the material instance there are some problems.
1) The animation doesn’t start from the start. Sometimes it appears to be halfway through playing. I’ve tried setting the Speed scalar to 0.0 initially, and changing this to 0.3 in the behaviour tree blueprint with a Set Scalar Parameter Value node, but this seems to introduce a long delay before the lip animation appears to start playing:
2) The play speed is inconsistent even if I set the Speed multiplier in the material to 0.3 for all material instances. I have Speed set to 0.3 by default since this matches the speed of the lip-sync animation with the NPC’s first line of dialogue. However, longer lines of dialogue animate faster, and shorter lines of dialogue animate slower. Is there any way to get the correct speed based on something like the frame length or duration or something? Perhaps there is there a better way to set up the Morph Animation pin on the MS_VertexAnimationTools_MorphTargets node?
Q. Why not just export the animation as a geometry cache from 3ds Max or Audio2Face and import it using the Nvidia Omniverse ‘Import USD Animation’ option?
A. I tried that and it fails to import. I can import it as an Alembic Geometry Cache from 3ds Max, however I’m a bit stoop and have no idea how to use it to drive facial animation, and if I import it as a skeletal animation instead it too fails to import. Also, based on this article I’m worried geometry cache will have too much of a performance impact. It also mentions vertex animation textures as a more performative alternative.
