Skeletal mesh: Kinematic and simulated coordinate systems mismatch (off by 90')

For a SkeletalMeshComponent, the bone transforms are internally stored in different coordinate systems (off by 90’), depending on whether the bone is kinematic or simulated. Whether or not this mismatch is intentional, it causes issues in several situations.

In pose record and playback at PhysX level:

The mismatch can be seen directly by following these steps for some skeletal mesh component: (pseudo)

// run simulated physics, record the pose sequence
SkeletalMeshComponent->SetSimulatePhysics( true );
for( int frame = 0; frame < nFrames; ++frame )
{
    for( int bone = 0; bone < nBones; ++bone )
    {
        buffer[frame][bone] =
            SkeletalMeshComponent->Bodies[bone]->GetPxRigidDynamic()->getGlobalPose();
    }
    < advance by one tick>
}

// switch off physics (make the bones kinematic), play back the pose sequence
SkeletalMeshComponent->SetSimulatePhysics( false );
for( int frame = 0; frame < nFrames; ++frame )
{
    for( int bone = 0; bone < nBones; ++bone )
    {
        SkeletalMeshComponent->Bodies[bone]->GetPxRigidDynamic()->
            setGlobalPose( buffer[frame][bone] );
    }
    < advance by one tick>
}

This works fine if the “SkeletalMeshComponent->SetSimulatePhysics( false );” line is commented out, as long as the skeletal mesh does not interact with (touch) other simulated actors.

However, if the skeleton is set to kinematic mode (the aforementioned line is not commented out), then the pose sequence plays back fine except being tilted by 90 degrees around the Y axis!

In the Physics Asset Tool (PhAT):

This seems like the likely cause for this bug in PhAT: https://answers.unrealengine.com/questions/102065/simulation-problem-in-phat-if-bone-is-marked-as-ki.html

This can be seen as follows:

  1. Repeat the steps mentioned in that post (open some simulated skeletal model in PhAT and set a few bones to kinematic)
  2. Place the skeletal mesh to the scene
  3. Hit ‘Simulate’ and immediately ‘Pause’, then advance the simulation frame by frame

The skeletal mesh becomes quickly completely corrupted, but it can be seen in the first few frames that some of the bones are making sharp 90’ jumps on each frame.

On a dedicated server with incompletely enabled simulation:

The rotation issue described in https://answers.unrealengine.com/questions/104567/enabling-skeletalmeshcomponent-physics-on-a-dedica.html is likely also caused by this. (See “The rotation issue” subheading.)

Engine versions: 4.5.0-0+UE4, 4.4.3-0+UE4 (source builds)

Hey ,

I’m trying to reproduce your issue internally, but I’m not extremely familiar with the interaction of animation and code. Can you tell me where you’re putting the code snippet you posted (so I can set up the repro case you’re describing)?

@JonathanDorman: Sure, I am using Owen from Content Examples and used roughly the following steps and code. I hope I didn’t forget anything crucial! You might want to omit the replication part and implement it as a standalone game, maybe.

  1. Start with a Minimal Code project
  2. Add code to project: extend from Actor, let’s name it ControlledRagdoll
  3. Migrate Owen from the Content Examples demo to your project
  4. Create a blueprint from the migrated Owen asset, reparent it to ControlledRagdoll, and place it in the world.
  5. In the settings tab of the blueprinted Owen, tick “Replicates” and “Replicate Movement”
  6. Crank up ConfiguredInternetSpeed, ConfiguredLanSpeed etc.
  7. (In world settings, disable gravity)

In YourProject.Build.cs, add PhysX stuff to the following to get direct access to PhysX:

PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "PhysX" });
PublicIncludePaths.AddRange(new string[] { "Engine/Source/ThirdParty/PhysX/PhysX-3.3/include" });

(see How to include the header file from a plugin - Plugins - Unreal Engine Forums)

Then, add the following to ControlledRagdoll.h:

(By the way, if someone knows how to replicate 3rd party data structs without this kind of a workaround, then please share! Thanks!)

#include <PxTransform.h>


USTRUCT()
struct FBoneState {
GENERATED_USTRUCT_BODY()


private:

UPROPERTY()
TArray<int8> TransformData;


public:

FBoneState()
{
	TransformData.SetNumZeroed( sizeof(physx::PxTransform) );
}

physx::PxTransform & GetPxTransform()
{
	return reinterpret_cast<physx::PxTransform &>(TransformData[0]);
}

};



UCLASS()
class RAGDOLLCONTROLLER45_API AControlledRagdoll : public AActor
{
GENERATED_UCLASS_BODY()


protected:

/** The SkeletalMeshComponent of the actor to be controlled. */
USkeletalMeshComponent * SkeletalMeshComponent;

/** Data for all bodies of the SkeletalMeshComponent, for server-to-client pose replication. */
UPROPERTY( EditAnywhere, BlueprintReadWrite, Replicated, Category = RagdollController )
TArray<FBoneState> BoneStates;

/** Store pose into the replicated BoneStates array. */
void sendPose();

/** Apply replicated pose from the BoneStates array. */
void receivePose();


public:

virtual void PostInitializeComponents() override;
virtual void Tick( float deltaSeconds ) override;
};

And to ControlledRagdoll.cpp:

#include <Net/UnrealNetwork.h>

#include <extensions/PxD6Joint.h>
#include <PxRigidBody.h>
#include <PxRigidDynamic.h>
#include <PxTransform.h>



void AControlledRagdoll::GetLifetimeReplicatedProps( TArray<FLifetimeProperty> & OutLifetimeProps ) const
{
Super::GetLifetimeReplicatedProps( OutLifetimeProps );

DOREPLIFETIME( AControlledRagdoll, BoneStates );
}



AControlledRagdoll::AControlledRagdoll(const class FPostConstructInitializeProperties& PCIP)
	: Super(PCIP)
{
// enable ticking
PrimaryActorTick.bCanEverTick = true;
}



void AControlledRagdoll::PostInitializeComponents()
{
Super::PostInitializeComponents();

/* init SkeletalMeshComponent: scan all components and choose the first USkeletalMeshComponent */

// get all components of the right type
TArray<USkeletalMeshComponent*> comps;
GetComponents( comps );

// if at least one found, choose the first one
if( comps.Num() >= 1 )
{
	this->SkeletalMeshComponent = comps[0];

	// warn about multiple SkeletalMeshComponents
	if( comps.Num() > 1 )
	{
		UE_LOG( LogTemp, Warning, TEXT( "(%s) Multiple USkeletalMeshComponents found! Using the first one." ), TEXT( __FUNCTION__ ) );
	}
}
else
{
	UE_LOG( LogTemp, Error, TEXT( "(%s) No USkeletalMeshComponents found!" ), TEXT( __FUNCTION__ ) );
}

if( this->Role >= ROLE_Authority )
{

	/* We are standalone or a server */

	//...

}
else
{

	/* We are a network client, ... */

	// Set the skeletal mesh to kinematic mode, so as to track exactly the pose stream received from the server (we do not want any local physics interactions or simulation)
	if( this->SkeletalMeshComponent )
	{
		// cannot do this, as UE appears to be using internally a different the coordinate system for kinematic skeletons!
		//this->SkeletalMeshComponent->SetSimulatePhysics( false );
	}
	else
	{
		UE_LOG( LogTemp, Error, TEXT( "(%s) Failed to switch SkeletalMeshComponent to kinematic mode on a network client!" ), TEXT( __FUNCTION__ ) );
	}

}
}



void AControlledRagdoll::Tick( float deltaSeconds )
{
Super::Tick( deltaSeconds );

// If network client, then apply the received pose from the server and return
if( this->Role < ROLE_Authority )
{
	receivePose();
	return;
}


/* We are standalone or a server */

// Store pose so that it can be replicated to client(s)
sendPose();

//...

}



void AControlledRagdoll::sendPose()
{
// check the number of bones and resize the BoneStates array
int numBodies = this->SkeletalMeshComponent->Bodies.Num();
this->BoneStates.SetNum( numBodies );

// loop through bones and write out state data
for( int body = 0; body < numBodies; ++body )
{
	physx::PxRigidDynamic * pxBody = this->SkeletalMeshComponent->Bodies[body]->GetPxRigidDynamic();
	if( !pxBody )
	{
		UE_LOG( LogTemp, Error, TEXT( "(%s) GetPxRididDynamic() failed for body %d!" ), TEXT( __FUNCTION__ ), body );
		return;
	}

	// Replicate pose, skip velocities
	this->BoneStates[body].GetPxTransform() = pxBody->getGlobalPose();
}
}



void AControlledRagdoll::receivePose()
{
int numBodies = this->BoneStates.Num();

// Verify that the skeletal meshes have the same number of bones (for example, one might not be initialized yet)
if( numBodies != this->SkeletalMeshComponent->Bodies.Num() )
{
	UE_LOG( LogTemp, Error, TEXT( "(%s) Number of bones do not match. Cannot replicate pose!" ), TEXT( __FUNCTION__ ) );
	return;
}

// (omitted: verify binary compatibility of the pose data)

// Loop through bones and apply received replication data to each
for( int body = 0; body < numBodies; ++body )
{
	physx::PxRigidDynamic * pxBody = this->SkeletalMeshComponent->Bodies[body]->GetPxRigidDynamic();
	if( !pxBody )
	{
		UE_LOG( LogTemp, Error, TEXT( "(%s) GetPxRididDynamic() failed for body %d!" ), TEXT( __FUNCTION__ ), body );
		return;
	}

	// Replicate pose, skip velocities
	pxBody->setGlobalPose( this->BoneStates[body].GetPxTransform() );
}
}

ps. There was a bug with reparenting at least in 4.4.3: “Save All” does/did not save reparenting information, instead you have to manually save the asset. Maybe you can file a bug report for this if this is still true with 4.5 and you think this is not too minor to be filed?

Hey -

I’m trying to follow your repro steps to git the same behavior on my machine and I have a few questions for clarification. What do you mean by “create a blueprint from the migrated Owen asset”? The Owen asset is a skeletal mesh so the only type of BP I can create from it is an AnimBP. Second, if I create an AnimBP and try to reparent it, it can only accept another type of AnimBP as the parent which means that ControlledRagdoll cannot be set as the parent. Did you mean for ControlledRagdoll to be an AnimBP as well? If you could further explain your reproduction steps it will help towards testing this issue.

Cheers

Hi ,

In UE4.6, Right click on the Owen skelmesh → Asset Actions → Create Blueprint Using This…

Hope this helps!

Hey -

I was working through your repro steps again and have a couple additional questions. I was able to create the blueprint based on Owen, however there are a number of compile errors due to the includes listed as well as pxBody (Specifically with the Get/Set BlobalPose() calls) in the source file. Are there any project specific files needed for this to compile? Also, in your repro steps, what do you mean in step 6 to “crank up ConfiguredInternetSpeed and ConfiguredLanSpeed”?

Did you add the mentioned lines to the YourProject.Build.cs file?

Engine.ini, which you can override with Config/DefaultEngine.ini, has the following default values:

[/Script/Engine.Player]
ConfiguredInternetSpeed=10000
ConfiguredLanSpeed=20000

Edit: these need to be adjusted too:

[/Script/OnlineSubsystemUtils.IpNetDriver]
MaxClientRate=15000
MaxInternetClientRate=10000

If you are going to reproduce this as a complete client-server system, then increase these so as to have a faster frame rate, but it is not essential for reproducting the actual issue. Also, you might want to completely omit the replication part and implement this as a standalone system, maybe.

I just tried to reproduce this in UE 4.6.1 and it works in my environment.

Two things: the PROJNAME_API keyword needs to match your project’s name, and the includes need to be arranged according to the instructions that the build system gives.

One thing that I forgot from the instructions is to enable physics for Owen (in which case you also want to disable gravity and maybe place Owen mid-air, to avoid collisions with the ground): enable Simulate Physics for the skelmesh component of your blueprinted Owen and set its collision class to PhysicsActor.

If you want I can send you the project files?

I added physics sim record feature in PhAT as well as Persona, and you can check what I did in 4.7. (big tool bar with “Record”

There is two issue with recording physics sim, and that is you’ll need the bone transform finally blended with physics, not before. Also I fixed an issue with root bone not matching (not identity) with phat, which might be partially issue.

The reason the record feature was added with 4.7 is because we also added double buffering system, where double buffering guarantees the buffer you query is what was last updated pose with physics blended.

You can check the code in 4.7. AnimationRecorder.h/cpp.

Thanks,

–Lina,