Neo Kinect - easy access to the Kinect v2 capabilities in your games

Someone asked some questions about how the Avateering demo Blueprints work on the Marketplace and I believe the answer might be useful to others, so I’ll leave it here as well, where I can link to and it’s easier to find (I think).

How the Avateering demo Blueprints work

I’ll call BP_AvateeringDemo and BP_AvateeringDemo_UE4 the Actor BPs, and AvateeringDemo_AnimBP and AvateeringDemoUE5_AnimBP the Anim BPs.

The Actor BPs communicate with the Anim BPs using functions from the BPI_AvateeringAnimBP Interface Blueprint. Both Anim BPs implement that interface to give meaning to its functions SetJointsTransforms and GetJointsTransforms. The Actor BPs get the AnimInstance from their SkeletalMeshComponents, which is the running Anim BP for a skeleton, and use those interface functions to send Transforms to it and also to get previously sent Transforms. By doing that, the Actor BPs don’t need to know which Anim BP specifically they’re talking to, meaning you can switch to other skeletons with different Anim BPs that implement that interface and the Actor BPs would still work with those skeletons, even if they’re 6 different skeletons, as long as they use an Anim BP that implements BPI_AvateeringAnimBP.

The Actor BPs are the only ones that talk to the Kinect sensor, reading Joints Transforms data from it. Then they get the previous frame’s Transforms sent to the Anim BPs using GetJointsTransforms and interpolate with the current frame data from Kinect, every frame. Then they send the interpolated Transforms back to the Anim BP using SetJointsTransforms. That way we can do noise filtering (smoothing) on Kinect’s joints data without needing to store an extra Transforms array for each mesh to read the previous frame data. We just use the array in each Anim BP.

The Anim BPs are the only ones that manipulate the Skeletal Mesh bones. I could’ve done it from the Actor BPs using nodes like SetSocketTransform on the SkeletalMeshComponents, but that’s much worse performance-wise. Besides, using the Anim BPs to manipulate the Skeletal Mesh bones enables the usage of other Anim BP functionalities, like jiggle physics, post-process Anim BPs, Anim Layers and so on. But, to reiterate, the Anim BPs just use the transforms they’re told to. The Actor BPs are the ones talking to Kinect, interpreting its Joints Transforms, then sending them to the Anim BPs.