Accessing Vertex Positions of animated skeletal mesh

Hi!

I’m trying to track the vertex position of a skeletal mesh during an animation but the doc about skeletal mesh api is very poor.

using SkMeshComponent->SkeletalMesh->GetResourceForRendering()->LODModels[0].GetVertices() I can access to the mesh pose vertices but I not sure how to access to the transformed vertices if the skeletal mesh is playing an animation.

any tip??

THanks!

I believe you would have to calculate the all the bone transforms and skin weights of the selected vertex and multiply manually.
This is because that multiplication happens in the shader (with GPU Skinning) when rendering so there is no way to access on CPU.

(If you turn off GPU Skinning, you may be able to access directly, but would incur the additional performance cost)

In Psuedo code,
Calculate which bones affect the desired vertex using the skin weights. (These are in the asset)
Get the ComponentSpace bone transforms from the RefSkeleton (these are cached every frame and reflect the animation)
Calculate the vertex postion.

Luckily, it looks like there might be a convenience function in MeshUtilities.cpp: CalcBoneVertInfos
(found in in Engine/Source/Developer/MeshUtilities)

It looks like this calculates all verts, so not sure if you want to run this every single frame…
but you could use this as a guide to roll your own “single vertex position calculater”.

If you do figure it out please post back - would be useful to have.

Good luck

Hi!

Thanks for your reply OptimisticMonkey. I have found the vertex position every frame with Skeletal->ComputeSkinnedPositions. It’s in component space but transform to world space is not hard. The problem is that this function returns just an Array of vectors. If I find the way to match its positions with the mesh vertices it will resolved. I want to create in runtime a static mesh from a specific frame of the mesh animation. Is each vertex in ComputeSkinnedPositions return in the same position than LODModels[0].GetVertices() returned array?? I don’t know, I will check it.

Bye!!

FYI - If All you want is to create a runtime static mesh - then there is the new PoseSnapshot capabilities in 4.15.

From Unreal Engine 4.15 Released! - Unreal Engine

NEW: POSE SNAPSHOTTING
We’ve added the ability to capture a runtime Skeletal Mesh pose in Blueprints for use in the Anim Blueprint!

Cool!

FYI - If All you want is to create a runtime static mesh - then there is the new PoseSnapshot capabilities in 4.15.

From Unreal Engine 4.15 Released! - Unreal Engine

“NEW: POSE SNAPSHOTTING
We’ve added the ability to capture a runtime Skeletal Mesh pose in Blueprints for use in the Anim Blueprint!”

Not sure if that is what you are looking for…

Hi!!

I didn’t know about this new feature!! thanks! I have been testing it but “save snapshoot” node from skeletalmesh has not a return so I dont know how to use this pose… I hope Epic give us more information about this feature soon.

Regard!

hey ! how you get that Skeletal->ComputeSkinnedPositions() worked , I’m getting some errors while passing LOD render data argument. saying “possible information loss”. could you please help me. bare with me if I’ve asked basic question.

Thanks.

This is how I do something like what you are asking. I use this to get the morphed mesh vertices from either the body mesh or the face mesh of MetaHumans. In the case of MetaHumans they contain multiple SkeletalMeshComponent, but if you want to pose the body/face then the control parameters must be set on the "Body" component e.g. face control parameters set on the "Face" component will not pose the face, set even the face parameters on the "Body" and then get vertices from the "Face" mesh.

TArray<FVector> UMyPlugin::SkeletalMeshComponentGetPosedVertices(
  USkeletalMeshComponent* ctrl_mesh,
  TArray<FName> curve_names,
  TArray<float> curve_values,
  USkeletalMeshComponent* posed_mesh,
  int LODIndex) {

  if (curve_names.Num() != curve_values.Num()) {
    UE_LOG(LogMyPlugin,
           Error,
           TEXT("Mismatch between length of curve names %d and values %d"),
           curve_names.Num(),
           curve_values.Num());
    return {};
  }

  if (const auto data = posed_mesh->GetSkeletalMeshRenderData();
      LODIndex < 0 || LODIndex >= data->LODRenderData.Num()) {
    UE_LOG(LogMyPlugin, Error, TEXT("Invalid LODIndex: %d"), LODIndex);
    return {};
  }

  // Clear all curve values to zero so that values that are not given are always zero.
  const auto _zero_curves = [&](EAnimCurveType type) {
    TArray<FName> names;
    ctrl_mesh->GetAnimInstance()->GetActiveCurveNames(type, names);
    for (const auto& name : names) {
      ctrl_mesh->GetAnimInstance()->AddCurveValue(name, 0);
    }
  };
  _zero_curves(EAnimCurveType::AttributeCurve);
  _zero_curves(EAnimCurveType::MaterialCurve);
  _zero_curves(EAnimCurveType::MorphTargetCurve);

  // Apply all curve values.
  for (int i = 0; i < int(curve_names.Num()); ++i) {
    ctrl_mesh->GetAnimInstance()->AddCurveValue(curve_names[i], curve_values[i]);
  }

  // Seems this magic needs to be called, otherwise mesh is not modified by control rig parameters.
  posed_mesh->GetAnimInstance()->InitializeAnimation();

  TArray<FFinalSkinVertex> verts;
  posed_mesh->GetCPUSkinnedVertices(verts, LODIndex);

  // Output only vertex position, not tangents or UVs.
  TArray<FVector> result;
  result.Reserve(verts.Num());
  for (auto& v : verts) {
    result.Add(FVector(v.Position));
  }
  return result;
}

I use this from Python code, but can be used from C++ or Blueprints. For example, to get a posed face:

root: unreal.SceneComponent = metahuman_actor.root_component
components = {c.get_name(): c for c in root.get_children_components(include_all_descendants=True)}
body: unreal.SkeletalMeshComponent = components['Body']
face: unreal.SkeletalMeshComponent = components['Face']
names = ['CTRL_expressions_jawOpen', 'CTRL_expressions_browLateralL']
values = [0.5, 1.0]
lod = 0
face_verts = unreal.MyPlugin.skeletal_mesh_component_get_posed_vertices(body, names, values, face, lod)

This code could probably be modified to allow passing in skeleton rotations to pose the body skeleton – currently it is only setting float curve values.

1 Like

Hello ColibriLive ,
so basically i am trying to get the vertex position (and normal) of a cloth that is continuously moving with wind, so it would be very useful if you have found a way to continuously get the vertex position of skeletal mesh.
Thanks in advance :slight_smile:

1 Like

use clothcomponent