I’ve imported a skeletal mesh over from 3ds Max in FBX format, and used Audio2Face to generate lip-sync animation which I’ve imported into Unreal as an Alembic Geometry Cache. Dialogue and subtitles are triggered via Behaviour Tree. Is there any way for me to play a specific geometry cache animation on my character via blueprint so I can animate the character’s lips while they speak?
If I drag and drop the geometry cache animation into the scene, I can see the lips move on the head, so the animation itself is fine. Just not sure how to go about setting it up to play on demand on the character, assuming that’s possible.
1 Like
I think you would have to spawn the asset in at the precise time you need it to spawn at.
And delete it afterwards before it loops back around.
Its not a particularly user friendly way to do things…
1 Like
Yeah I feel like I might be heading down the wrong path trying to import the animation in Alembic format.
The way that the animation should be imported is via the Omniverse plugin’s right-click ‘Import USD Animation’ option since this is coming from Audio2Face.
However this is causing the editor to crash.
Excerpt from the log file:
[2024.05.03-04.41.31:681][607]LogSlate: Window 'Omniverse USD Animation Importer' being destroyed
[2024.05.03-04.41.33:132][607]LogWindows: Error: === Critical error: ===
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: Fatal error!
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x0000000000000000
[2024.05.03-04.41.33:132][607]LogWindows: Error:
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcef8ca350 usd_gf.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffceac37091 usd_usdSkel.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffceac333d9 usd_usdSkel.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea5e76db UnrealEditor-OmniverseUSD.dll!USDConvert() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\USDConverter.cpp:5191]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea442c0d UnrealEditor-OmniverseUSD.dll!<lambda_26744bb5f100f2393189e68d24e4e65e>::operator()() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:271]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea477b73 UnrealEditor-OmniverseUSD.dll!OmniverseAnimationImporterImpl::ImportSkelAnimation() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:290]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea48deff UnrealEditor-OmniverseUSD.dll!OmniverseAnimationImporterImpl::LoadUSDAnimationInternal() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:409]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcea48d735 UnrealEditor-OmniverseUSD.dll!FOmniverseAnimationImporter::LoadUSDAnimation() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseUSD\Private\OmniverseAnimationImporter.cpp:430]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffce0c686be UnrealEditor-OmniverseEditor.dll!SOmniverseImporterDialog::OkClicked() [D:\b\0\_build\OmniConnectorBuild\Plugins\Omniverse\Source\OmniverseEditor\Private\OmniverseImporterDialog.cpp:136]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffce0c4475a UnrealEditor-OmniverseEditor.dll!TBaseSPMethodDelegateInstance<0,SOmniverseImporterDialog,1,FReply __cdecl(void),FDefaultDelegateUserPolicy>::Execute() [D:\b\0\_build\target-deps\UnrealEngine\Engine\Source\Runtime\Core\Public\Delegates\DelegateInstancesImpl.h:275]
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18fc3f66 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18fea355 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18d7b552 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18d602b8 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18dc89b3 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18db7430 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd18daccf0 UnrealEditor-Slate.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb50b12 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb36751 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb533c8 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb2ff43 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56bef75 USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56be9ee USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffcd59c7f09 InkObj.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd70f210b8 atlthunk.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56bef75 USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda56be69d USER32.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffd8bb54856 UnrealEditor-ApplicationCore.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a71750b UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a73473c UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a73482a UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a7374f0 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a74a4c4 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ff61a74d996 UnrealEditor.exe!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda59a7344 KERNEL32.DLL!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error: [Callstack] 0x00007ffda61026b1 ntdll.dll!UnknownFunction []
[2024.05.03-04.41.33:132][607]LogWindows: Error:
Interestingly, if I drag and drop in the .usd file containing the blendshapes exported from Audio2Face instead of using the right-click menu option, it recognizes that there is animation in that file:
However this method also fails to import (without a crash). So that’s why I’ve been looking for an alternative way of importing the facial animation. Basically I tried following this video to import the animation even though the video is geared around Metahuman and I’m working with a custom rig.
2 Likes
Tried exporting the Geometry Cache from Audio2Face in Maya format so I could bring it into 3ds Max first. In 3ds Max I can apply the Geometry Cache successfully to my face mesh using the Point Cache Binding modifier. If I scrub the timeline, the animation plays fine. I then exported the face mesh in USD format with Use Animation and Use Animation Range ticked:
Unfortunately I still can’t import it into Unreal Engine with the Omniverse right-click option ‘Import USD Animation’. I get a ‘Failed to Import’ message and the logfile shows the following:
[2024.05.06-01.49.38:454][879]LogOmniverseImporter: Error: Skeleton c14-TestRig_Skeleton isn't compatible with C:/Users/Knut99/Desktop/c14-TestRig/a2f/TestRig_GeoCacheExportedFrom3dsMax.skelanim.usd
Back to trying to work something out with Alembic, I guess?
2 Likes
So I can import an Alembic as a skeletal animation by changing the Import Type to Skeletal like so:
If I go into the Animation Sequence, the face animates as expected. I created a montage then tried to get my character to play the animation via blueprint in the Behaviour Tree (since this is for lip-sync animation during dialogues) but my character just t-poses. I’m guessing there’s some sort of skeleton mis-match happening here? I have no idea. I’ve tried:
- Exporting just the face mesh with the Point Cache modifier from 3ds Max in Alembic format.
- Exporting the face mesh with all bones from 3ds Max in Alembic format.
- Exporting all meshes with all bones from 3ds Max in Alembic format.
Same problem each time. My character just t-poses. This was exported from the same .max scene that I used to bring the character rig into Unreal Engine originally in FBX format, so there shouldn’t be a bone mismatch.
The FBX format supports point cache, however it doesn’t seem like Unreal supports importing point cache from FBX, which is unfortunate. Gave it a shot anyway, but as expected no animation is imported. If I import the FBX back into 3ds Max I get the point cache successfully, so the point cache animation is definitely in the FBX file. Basically created a Selection Set including the bones and the face mesh, exported to FBX and specified the Selection Set.
