Accessing Vertex Positions of animated skeletal mesh

Development Programming & Scripting C++
9

This is how I do something like what you are asking. I use this to get the morphed mesh vertices from either the body mesh or the face mesh of MetaHumans. In the case of MetaHumans they contain multiple SkeletalMeshComponent, but if you want to pose the body/face then the control parameters must be set on the "Body" component e.g. face control parameters set on the "Face" component will not pose the face, set even the face parameters on the "Body" and then get vertices from the "Face" mesh.

TArray<FVector> UMyPlugin::SkeletalMeshComponentGetPosedVertices(
  USkeletalMeshComponent* ctrl_mesh,
  TArray<FName> curve_names,
  TArray<float> curve_values,
  USkeletalMeshComponent* posed_mesh,
  int LODIndex) {

  if (curve_names.Num() != curve_values.Num()) {
    UE_LOG(LogMyPlugin,
           Error,
           TEXT("Mismatch between length of curve names %d and values %d"),
           curve_names.Num(),
           curve_values.Num());
    return {};
  }

  if (const auto data = posed_mesh->GetSkeletalMeshRenderData();
      LODIndex < 0 || LODIndex >= data->LODRenderData.Num()) {
    UE_LOG(LogMyPlugin, Error, TEXT("Invalid LODIndex: %d"), LODIndex);
    return {};
  }

  // Clear all curve values to zero so that values that are not given are always zero.
  const auto _zero_curves = [&](EAnimCurveType type) {
    TArray<FName> names;
    ctrl_mesh->GetAnimInstance()->GetActiveCurveNames(type, names);
    for (const auto& name : names) {
      ctrl_mesh->GetAnimInstance()->AddCurveValue(name, 0);
    }
  };
  _zero_curves(EAnimCurveType::AttributeCurve);
  _zero_curves(EAnimCurveType::MaterialCurve);
  _zero_curves(EAnimCurveType::MorphTargetCurve);

  // Apply all curve values.
  for (int i = 0; i < int(curve_names.Num()); ++i) {
    ctrl_mesh->GetAnimInstance()->AddCurveValue(curve_names[i], curve_values[i]);
  }

  // Seems this magic needs to be called, otherwise mesh is not modified by control rig parameters.
  posed_mesh->GetAnimInstance()->InitializeAnimation();

  TArray<FFinalSkinVertex> verts;
  posed_mesh->GetCPUSkinnedVertices(verts, LODIndex);

  // Output only vertex position, not tangents or UVs.
  TArray<FVector> result;
  result.Reserve(verts.Num());
  for (auto& v : verts) {
    result.Add(FVector(v.Position));
  }
  return result;
}

I use this from Python code, but can be used from C++ or Blueprints. For example, to get a posed face:

root: unreal.SceneComponent = metahuman_actor.root_component
components = {c.get_name(): c for c in root.get_children_components(include_all_descendants=True)}
body: unreal.SkeletalMeshComponent = components['Body']
face: unreal.SkeletalMeshComponent = components['Face']
names = ['CTRL_expressions_jawOpen', 'CTRL_expressions_browLateralL']
values = [0.5, 1.0]
lod = 0
face_verts = unreal.MyPlugin.skeletal_mesh_component_get_posed_vertices(body, names, values, face, lod)

This code could probably be modified to allow passing in skeleton rotations to pose the body skeleton – currently it is only setting float curve values.

1 Like