First off kudos to the team! The new animation work flows are insanely good and incredibly impressive and powerful. I have been able to get amazing quality facial motion captures, it is opening up new possibilities for production, and increasing what is possible for smaller teams. So a super heartfelt thank you to the team for making such amazing tools.
That being said, with this new work flow I am noticing some issues that are coming up as I am getting Meta Human Performances into my characters system. Specifically around the tediousness of tracking a dialogue and Meta Human performance as a separate element creates production issues and decreases dev quality of life.
For example, if my performance take is on the longer side-- ie. a read through of an entire scene, the performance can be pretty long-- around 20 minutes. If I want to keep track of the dialogue, it is pretty tedious to use notifies (but it is possible). For clarity sake here is a simple example-- I have a response starting 00:07:54 in my animation, I have to figure out the starting time for the audio cue to play the audio in the location of the actor-- and place additional custom notifies in the time line with the section markers in the Anim-Montages-- this adds up fast…
But it would be totally possible and pretty easy if not tedious by adding the notifies in the Anim-Montage sequence…
Major issue editor reliability:
However the play back system in the Anim-Montage editor is pretty buggy and has been causing a lot of crashes and weird errors. For example, if I add in a simple “play sound” notify in the Anim-Montage, the sound doesn’t stop when I stop the editor, yet it will play again if I hit play-- stacking the sound and creating chaos until I shut down the editor. (a bug) Further, if you play the animation after the sound-notify the sound will not play because you didn’t trigger the notify. (Kinda a design issue) So even in the Anim-Montage editor, getting the sound to play in sync with the performance can be challenging-- making it difficult to place the animation notifies as there is no synced sound reliably in the editor. So just navigating and scrubbing the time line with audio is weirdly difficult. (This was typically not the issue with anim-montages as they tended to be very short)
Solution 1: Change my work flow to have short takes-- while that may work, it doesn’t really fix the issue as i want many possible responses in the Anim-Montage so I can navigate the time line via the section selector-- so even if the Animation sequences are shorter I would still need lengthy-montages made up of multiple Anim-sequences.
Solution 2: A special Synced-Animation class that would be for this type of scenario, where you always want the animation to be played in coordination with a synced sound, so when ever the animation would play, the animation would also play the sounds on the timeline like in a traditional editing software. So rather than using the notify system which is great for many use cases, these animations would have audio that plays along via a timeline and makes syncing audio easier, for animations that have this specification.
Personally, I think solution 2 would be a super useful system and really really open the work flow to less technical developers, as it is the current work flow keeping things in sync is surprisingly non intuitive-- especially coming from a film editing background.
Anyway-- bravo on the meta human animation systems-- totally incredible, but this is something that would be a very useful addition and help with dev-qol.
Also I could totally be missing something-- I am not using the Dialogue Wave system (yet) so maybe there is a much easier system that I am just missing so please let me know if there is an easier way to track audio and animation performances-- that system seems great for localization but not exactly what I am talking about unless I am overlooking features.
