UE 5.8.3 MetaHuman From Custom Mesh (Head Only): FaceMesh stays archetype-identical when facial tracking fails, though native ConformTarget does deform — what input contract are we missing?

Development Character & Animation
, , ,
1

Epic Issue Draft — post manually (user action required)

Free channel (UDN requires a paid support plan): Post in the official Epic forums → Character & Animation → MetaHuman area (https://forums.unrealengine.com — free Epic account). Body = this document; attach every small file from this epic_repro/ folder. Alternative: Epic Developer Community MetaHuman discussions.

Title (selected, Candidate A): UE 5.8.3 MetaHuman From Custom Mesh (Head Only): FaceMesh stays archetype-identical when facial tracking fails, though native ConformTarget does deform — what input contract are we missing?

Environment

  • Unreal Engine 5.8.3 (5.8.3-58210709+++UE5+Release-5.8), Windows 11 26H2
  • MetaHuman plugin 1.0.0 (engine-installed, unmodified)
  • CPU i5-13600KF, GPU RTX 4070 Ti SUPER (driver current), NNERuntimeORT present
  • No engine or plugin source modifications; only public APIs and the official editor UI used

Description

We are evaluating the in-engine MetaHuman Creator Import → From Custom Mesh → Head Only workflow with an AI-reconstructed head mesh (single-photo reconstruction, 15027 verts, eyes present, quality-gated). The mesh is accepted, the solve runs, and the character assembles successfully — but the final FaceMesh is byte-identical to the archetype face (SHA256-identical OBJ exports; per-vertex delta exactly 0.000000 across all 18649 vertices).

Our machine cannot produce automatic facial landmarks: the NNE face detector returns no detections on both the default NNERuntimeORTDml and a forced NNERuntimeORTCpu backend (mh.FaceTracker.Backend), for real portrait photos and rendered meshes alike (log: TrackFaceLandmarksFromImage: no face landmarks detected). In the GUI, Auto Solve reports Failed to track face landmarks from auto-framed image and the documented manual fallback (Trace Facial Features) fails the same way.

When tracing fails, the geometric solve continues and reports success — the resulting character simply carries no identity.

As a control, calling the public FMetaHumanCharacterBodyIdentity::FState::ConformTarget (MetaHumanCoreTechLib) with 3 KeyPointTargets on the same target mesh produces a real head-region deformation: 12337 of 73823 vertices changed, max displacement 1.0202 cm, A/B hash distinct from the no-keypoint control arm. So the native solver stack does respond on this machine; the identity never reaches the final FaceMesh through the official Character Editor path.

Reproduction Steps

Headless (attached repro_headless.py, ~6 min, uses only official subsystem APIs):

  1. Import the custom head mesh (head_only.obj attached) via AssetImportTask.
  2. Create a fresh UMetaHumanCharacter (factory-new), CommitArchetypeFaceDNA for a buildable baseline.
  3. Call ConformToTargetMeshes (the same UMetaHumanCharacterEditorSubsystem function the GUI Auto Solve button invokes through UMetaHumanCharacterEditorMeshImportToolProperties::StartMeshConform) with TargetPartsType=HeadOnly, bAutoSolve=true, PipelineName="head_only", and the mesh’s vertices/indices from GetMeshDataForConforming. (On this machine the automatic tracing this call would normally rely on is unavailable — see Description.)
  4. RequestTextureSources (blocking), BuildMetaHuman (assembly succeeds).
  5. Export the built FaceMesh (GetMeshDataForConforming on the Unpacked SKM FaceMesh).

GUI (same result):

  1. Open a factory-new MetaHuman Character asset → Mesh Import tool → select head_only → Head Only → Auto Solve.
  2. Observe Failed to track face landmarks from auto-framed image (three occurrences across two Auto Solve invocations); mesh solving continues.
  3. Complete and build the character; export the FaceMesh.

Expected Result

The resulting MetaHuman FaceMesh should preserve the facial identity of the supplied custom head mesh (or, at minimum, differ from the archetype face).

Actual Result

The custom mesh is accepted and the MetaHuman character assembles successfully, but the final FaceMesh remains byte-identical to the archetype face:

text

archetype_facemesh_reference.obj  SHA256 9529f7a8d1efa0aa8282f856b29f73a3fe00759ec01742e8a3c3028d085f7f05

gui_autosolve_facemesh.obj SHA256 9529f7a8d1efa0aa8282f856b29f73a3fe00759ec01742e8a3c3028d085f7f05

per-vertex delta: 18649/18649 vertices at 0.000000 cm (max = mean = p95 = 0)

Evidence (attached)

Experiment Path Input Solver/Build Final FaceMesh
A GUI Auto Solve (2 invocations) custom mesh tracing failed ×3, geometric solve ran, assembly success archetype (SHA256 match)
B python ConformToTargetMeshes — 5-arm controlled trial (no keypoints / 3 correct / out-of-range indices +30 cm / in-range indices 0-2 +30 cm / cloud rotated +90°) same mesh solve success ×5 byte-identical archetype ×5
C public native FState::ConformTarget A/B 3 keypoints native conform 12337 verts changed, max 1.0202 cm, AB_IDENTICAL=0
D native, calibrated keypoints + official 102-point preset schema (via GetPresetBodyKeyPoints / CommitTargetMeshKeypoints data path) 7 semantic points causal response (EXP≠PROBE≠CONTROL) deformation not target-directed; editor-path FaceMesh still archetype

Attachments: head_only.obj, both FaceMesh OBJs, repro_headless.py, repro_native_receipt.md (driver excerpt + raw receipt), log excerpts (logs/).

Questions for Epic

Q-A — input contract. For UE 5.8.3 MetaHuman Creator From Custom Mesh / Head Only: when automatic facial landmark tracking is unavailable, what is the officially supported way to provide the required identity constraints to the conform solver? Is there a supported input form (semantic keypoint schema, curves/correspondences, camera+pose state) that does not depend on the NNE face detector?

Q-B — HeadOnly transfer behavior. Why does public FState::ConformTarget with KeyPointTargets produce measurable head deformation, while the Character Editor / From Custom Mesh path produces an archetype-identical FaceMesh? Which prerequisite along ConformTarget → FitWithVertexDeltasFromBody → FaceState → Build are we missing — e.g. CurveTrackingPoints, camera view info, Identity pose state, target-mesh keypoint schema, HeadOnly initialization — or is a tracing-derived constraint mandatory for any identity transfer?

Notes

  • We are not claiming a TITAN bug: the native control (Experiment C) shows solver responsiveness. What we cannot find is the officially supported data path from a custom mesh (without working auto-tracing) to an identity-bearing final FaceMesh.
  • The web MetaHuman Creator is discontinued (UE 5.6+ in-engine Creator migration), so the in-engine path above is the current official route for us.