Nanite Skeletal Mesh vertex stretching when using LODs/LOD Bias

Edit:

I checked in Vanilla 5.8.3 and was not able to reproduce this behavior on either the SKM_Quinn_Simple or our own meshes.

Which begs the question:

Is there a fix in 5.8 you can point us to that we can integrate?

----

We are in the process of switching our characters over to utilize Nanite Skeletal Meshes. During this process, some of the characters are still using non-Nanite skeletal meshes, and we have an existing r.SkeletalMeshLODBias setting on lower quality modes to reduce polygonal density for performance reasons.

We intend to have more than one LOD on our Nanite Skeletal Meshes so that we can define a lower density LOD level as the Ray Tracing Min LOD.

We have noticed that when r.SkeletalMeshLODBias is > 0, Nanite skinned mesh instances with sufficiently reduced LOD density show vertices stretched towards the object origin.

From the 5.7.4 source, the cause appears to be that:

  • USkinnedMeshComponent::UpdateLODStatus applies the bias to Nanite skinned components.
  • FDynamicSkelMeshObjectDataNanite::Init passes that biased LOD to UpdateRefToLocalMatrices.
  • UpdateRefToLocalMatrices only fills ReferenceToLocal for that LOD’s ActiveBoneIndices (all other bones end up as Identity times the inverse bind matrix).

Since Nanite skinning always uses LOD0 weights (per the comment in UpdateBonesRemovedByLOD), any bone dropped from the biased LOD during reduction skins LOD0 vertices with an unset matrix and collapses towards the object origin. UpdateBonesRemovedByLOD covers bones in the explicitly defined “BonesToRemove” list but not bones removed implicitly by reduction. In the repro case above, reducing LOD2 to 2000 triangles is what appears to drop the finger bones.

Two questions:

  1. Is this a known issue, and is there a fix in 5.8 we can integrate?
  2. What is the intended behavior? Should the ReferenceToLocal matrices used for Nanite cluster skinning always be built from LOD0’s ActiveBoneIndices (keeping the biased LOD only for RayTracingLODIndex), or should r.SkeletalMeshLODBias not apply to Nanite skinned components at all? We would like to implement a local fix in the meantime and have it match your recommendation.

[Image Removed]

[Attachment Removed]

Steps to Reproduce

In a 5.7.4 project with Third Person content added, open up SKM_Quinn_Simple in the Skeletal Mesh asset editor. Make sure r.Nanite.AllowSkinnedMeshes = 1.

  1. Set SKM_Quinn_Simple’s LOD2 Max Triangle Count to 2000.
  2. Place SKM_Quinn_Simple in a level.
  3. Set SKM_Quinn_Simple’s Enable Nanite Support to True.
  4. In the console, set r.SkeletalMeshLODBias 2.

Observe stretching on the Nanite skeletal mesh in the level viewport.

See attached images for example. [Image Removed]

[Attachment Removed]

I ended up integrating this change:

Fixed inverse refpose being applied to an identity transform when bon… · EpicGames/UnrealEngine@8682dfc

It appears to mitigate the issue by setting affected vertices to their bind pose position instead of the object origin. There is still stretching if the selected LOD is too reduced in comparison to the Nanite mesh, but at least it’s not to the origin.

[Attachment Removed]

Yeah, that would be the one that we would suggest, but you’re indicating that everything is not 100% fixed for you. Do you have a repro that you could provide for a better fix?

Dustin

[Attachment Removed]

Looking at the code, it seems it’s working as intended. Instead of setting vertex positions to object origin, it seems to be setting them to the bind pose.

[Attachment Removed]