Kinect 4 Unreal

Development Programming & Scripting C++
, , ,
62

Thank you very much :slight_smile:

63

Iparra:

We took a look at that sort of functionality and we currently don’t have any plans to go in that direction. We are planning concentrating our efforts towards increasing fidelity in sensor tracking, making the current features better and finding ways to include the further Kinect libraries.

64

Ok, thanks!

65

Hello Opaque Multimedia.

First of all, thank you for your plugin, it is awesome :slight_smile:

As the 4.7 version of UE came out a week ago, are you planning an update on your plugin ? If yes, when ?

66

Hey everyone, now that GDC is over we are able to release our version 4.7 versions of the plugin. Some of you came to see us and give your feedback on the plugin which was great and it has been fantastic to get to meet some of our community in person.

Hot Fix for Version 4.7
Hot Fix for Version 4.7 with introduction

67

Amazing work! Is the plugin available also for mac?

68

Do you have any suggestions on how to implement a kalman-filter or something similar for the kinect input data? I have too much jitter on my arms and hands when i map the data on a skeleton.

69

These links are 404ing for me. Did anyone get the files before they were taken down?

Edit: The links directly on / are working

70

Hey,

is it only possible to use the plugin with blueprints? I’m currently working in a c++ project and want to use Kinect.

71

Thanks! The plugin unfortunately is Windows only. This is due to the fact that the Kinect device is for Windows machines only. There might be a way to get a solution working through a dual boot setup on your mac if that is a possibility.

We have a simple suggestion you can try (assuming you are driving everything through the Animation Blueprint). On a per-bone basis you could try using the interpolation nodes Unreal provides (://docs.unrealengine.com/latest/INT/BlueprintAPI/Math/Interpolation/RInterpTo/index.html). Below is an example of a starting node setup in the Anim Graph followed by the changes needed to using the Interpolate node (with your own configuration for interp speed etc).

Starting Anim Graph:

Starting.PNG
Starting.PNG962×155 45.1 KB

Modified Anim Graph:
AnimGraph.PNG

Modified Event Graph:

EventGraph.PNG
EventGraph.PNG964×320 53.1 KB

If you would like to implement a more advanced smoothing approach, we would recommend you investigate using Blueprint Macros for calculations (they keep state and can have an internal array of values) or implementing a solution in C++ logic that takes values retrieved from the plugin functions.

The plugin exposes the Kinect data as blueprint nodes alone for the moment. This was a deliberate decision by us to ensure that our plugin could be as usable as possible. Have you tried extending a C++ class into blueprints and then using a Kinect Interface Component to retrieve the data that is then processes by your C++ logic? Something beyond what is outlined is this: ://docs.unrealengine.com/latest/INT/Gameplay/ClassCreation/CodeAndBlueprints/index.html

72

uhm you mean I should for example have a variable HeadLocation in c++ code, then get the location of the head in blueprints and set the C++ variable from the blueprint?

73

Btw I noticed that when I use the poseable mesh, it doesn’t change handstates on the character. How can I add this?

74

Essentially yes for the code integration.

As for the hands, the Kinect does not currently retrieve bone information for fingers so we cannot dynamically set their rotations. It is relatively trivial to make hand states trigger blended animations for the hands, however each use case for this is too different to provide just one solution. The models set up of hand bones and what exactly each hand state looks like changes for every implementation.

In addition to that, the Poseable Mesh is more of a quick drag and drop solution. The main way we have done hand animations on our projects is to check the hand state and then selective blend in a hand animation using blend nodes in the animation blueprint.

Also the problem with the links is now fixed, there was a mistake in our naming convention.

75

Thanks for the blend node tip, I managed to blend hand animations, but how do I add these animations to the poseable mesh? As far as I can see there’s no option to add an animation blueprint to the poseable mesh.

edit: I’m trying to rotate a character’s joints based on the output from get converted joint orientation and I managed to do this for one joint. I could repeat everything for every bone but then my blueprint would get very messy. Is there a way to do this for every possible joint?

ddc5ad4d78b12353bd45fefde1c583d8b7ad9f27.jpeg
ddc5ad4d78b12353bd45fefde1c583d8b7ad9f27.jpeg909×271 63.2 KB

40771d191f.jpg
40771d191f.jpg1123×215 68.3 KB

Btw, are you gonna add voice recognition to your plugin?

76

@
Unfortunately a poseable mesh component has much more limited feature set than what is possible using Animation Blueprints to control a Skeletal Mesh Component. We Kinect Poseable Mesh Component to be a simple drag an drop solution for basic avateering. We have found that a more complex avateering solution is just not possible with it and will require you to dive into the animation blueprint.

In regards to your Animation Blueprint implementation in the attached images, you will need to apply the joint rotation for every possible joint. Below is a suggestion for how you might be able to achieve that in a slightly cleaner way.
We create a target body on the Animation Blueprint and set it in the Blueprint Event Graph -

BlueprintSetup.PNG
BlueprintSetup.PNG1265×286 66.6 KB

We then use that target body variable directly for retrieving and converted rotations and feeding them into the transform bone nodes -
AnimBPSetup.PNG

These screenshots were taken from the BP_Avateering_Anim and the ABP_Avateering_Anim assets in our Kinect 4 Unreal Introduction project which is now up on the Marketplace as well!

I hope this helps you get your avateering solution where you would like it to be :slight_smile:

77

Thanks for the reply and for the introduction project. Very helpful :).

78

The plugin works great. But I spent the last two hours finding a solution to integrate it within a c++ project. But I always end up with error : Couldn’t find referenced module ‘Kinect4Unreal’

I understand that this error is because the compiler is looking for a Source folder within the project to compile as well. But is there really no workaround for this. Can’t you provide an empty Module that links to a lib file instead or something?

79

Hey,

I saw you guys recently pushed up an update. I was wondering were there any improvements made to the avataring? My project heavily uses it and I am curious to know if it is worth spending the time upgrading my project (I’m near to release so I don’t want to unless it is really worth it).

Cheers

80

I get the same error when trying to package my project

81

Yes that is the same problem. It tries to compile the code for the plugin but can’t find any…