Kinect 4 Unreal

K4U_LogoThin.png
1.1 by Opaque Multimedia

Over the past few months we have been hard at work creating a simple plugin that exposes all of the functionality of the Kinect 2 to Unreal Engine 4. Now we are proud to have our first public release of the plugin.

Kinect4Unreal is a plugin that exposes all of the Kinect functionality to Blueprints in Unreal Engine 4. We’ve focused on making the Kinect as easy to use as possible, so that artists and designers don’t need to have a coder on hand in order to experiment with the power of the Kinect.

Documentation

Download Links:
Plugin Version 1.1c for Unreal Engine version 4.10

Check our website for legacy versions.

Unreal Engine 4 Marketplace Content:
As announced in the recent post on the Unreal Engine blog we have now released our content projects on the Unreal Engine 4 marketplace. The projects available are:
Kinect 4 Unreal Introduction
Kinect 4 Unreal Example Levels

Previous versions (compatible with older versions of unreal) can be downloaded at our downloads page.

Also you can look for us on the marketplace soon!

**If you want a quick overview of the most (but by no means all) of the basic features:
**

://www.youtube.com/watch?v=WHmPvZvRyxc

While raw Kinect data is all well and good, it almost always needs significant processing to be truly usable. We have done a lot of that work for you, and give you access to a host of usability-oriented features that streamline kinect development. Here are just some of the tools we give you:

If you want a quick run down of how easy all this is to set up and use, this video goes from starting a blank project to deriving hand locations in 10 minutes.

Although this was slightly before our public release, had its first deployment at PAX Australia with an avateering set up of Ruby from Roster Teeth’s RWBY.

If you want more information then please visit our website or just post below.

Also look forward to future releases where we have such improvements as inverse kinematic based avateering, face information values and much more.

This looks really promising! I’m definitely going to check it out today.

Tried this plugin yesterday, and it’s working pretty nicely. I would have some issues with the licensing conditions, though, especially the royalties. For one, I don’t like the idea of having to report my revenues to numerous companies. And, at least for what it does today, the licensing costs are too high. If you really add something like IK avateering with a full-body solver those prices might be justified, but not right now (but even then, using something like IKinema and maybe the open source Kinect plugin that lionhead did would be a valid and cheaper alternative as well).

Anyways, I’ll certainly keep using this for my prototype right now and see how it goes, and then weigh the options later on.

Hi, this is a great plugin. However, I am quite new to Unreal Engine and having trouble packaging the plugin with my project. Please any help will be appreciated. Thanks

Rastar:
We’ve gone with a profit-royalty based model so that we’re only charging you if your project makes a profit. We don’t want to levy onerous upfront charges or to demand revenue, we feel this model allows more people to make better projects with the plugin, then you don’t lose anything if your project doesn’t work out.
We understand reporting revenue can be a pain, so we installed a royalty free threshold so you don’t have to worry about reporting revenue if your project makes less than $10,000. You can be assured that we hold any information you transmit to us in strictest confidence – if you have any privacy or other concerns, email us at [EMAIL=“k4u@.com”]k4u@.com and we’d be happy to work with you.

Ehis:
On packaging the plugin there is currently a bug in unreal that prevents packaging while using plugins. Epic are working their hardest to fix it. We expect packaging plugins to be available sometime in the early parts of next year. If you’re working on an installation project, you may be able to get by without packaging – we’ve done several installations ourselves, we can help you there if you need it.

Yes I would really appreciate it if I could get some help with installation.

Can somebody uplaod a complete scene (project) with a rig controlled by kinect? I can’t figure it out on my own. My orientations are somehow wrong. It would really help!

Hi,

Thanks for this… i hope @Unreal Engine 4 staff, make will permanent include from Unreal Engine 4. Also add FBX files to use in animation for HeroTPP or any Mannequin. Its give benefit for all; less expensive animation or less time for creating animation…More power to all :slight_smile:

I also think we would like to see a rigged 3D character usable for Kinect! I hope soon!

Ehis:
The simplest way to run an installation is to operate your project as a new editor window or standalone game from within the editor. Both of these will allow for complete resolution and full screen control as though it were a packaged executable. Once during a talk with Epic it was implied that turning the project into a code project would allow it to compile however our tests have shown that this does not work reliably, if at all. We are still looking for other solutions in the mean time though.

Viewtron, Peude and iparra:
The buzz around avateering is actually quite exciting for us and avateering is a high priority feature for us - as a part of the roll out for that, we’ve updated our introduction level to include a basic avateering demonstration that should allow you to see how things are put together and get you all on your feet (literally). When our content examples enter the marketplace they will also contain several in depth examples of each of the various different methods that can be used to achieve avateering and some of the things that can be built on top of it.

For your convenience here is the download link for the plugin and introduction content again. This is identical to the previous link but both now contain avateering.

Just set up a basic blueprint using this plugin and it looks very promising. Nice work, will keep you posted.

this looks interesting to say the least, but can’t use it because of UE4.6, is there any information on a new (recompiled) release?
Edit: I just noticed 4.6 is the latest over 4.5 (I reinstalled UE this week, wasnt aware I specifically installed a very current release), installed 4.5.1 for this, works like a charm, very sexy stuff. however my question is till valid :stuck_out_tongue:

I need help. I tried to rebuild anything with my own rig, but somehow it’s not really working, could someone explain what I’m doing wrong?

This is my test project:
://dl.dropboxusercontent.com/u/66937887/rigtest.zip
(“Knet” is my rig, ingore the standard ue4 rig)

EDIT: Ok got it working, forget to assing the animation blueprint to the skeletalmesh.

Why are all rotations of the bones wrong? I tried 3ds Max and Blender both are not working correctly.

BluE:
We are please to present the 4.6 Hot Fix version of the plugin. Previously we had plans to release version 1.1 that also contains several new features as well as 4.6 compatibility soon, however we want to have all of our features fully tested before release. So in the mean time we have created a version of the 1.0 plugin that is 4.6 compatible.

Viewtron:
By default we decided to have our plugin work with the “standard unreal skeleton” such as the blue man (Hero/Tutorial TPP) and owen. We had a lot of discussions about which skeleton to support by default and because there is no real standard skeleton in the world we chose to go with what unreal was using. But we also recognize that not everyone will want to use this skeleton and have created ourplugin so that inside the anim blueprint you can access the rotators and set them up manually. In general this is not a pleasant experience regardless of the tool or system that you are using. Shortly (as in I will likely be editing this post shortly) we will be going to release an example for the blender skeleton (MetaRig) set up for avateering in our plugin and hope to in future release tools to make this experience easier.

Also a note: in an upcoming version we will be changing the name of a node in the avateering blueprints; the Joint Orientation function in the anim blueprint is modified on a per joint basis to work with the unreal skeleton and hence is different from the other Joint orientations and will now have a different name. This should not cause people too many problems unless they are setting up a custom skeleton in which case you will have to recreate several nodes. Please be sure to keep records of any working rotation set ups so you minimise your own workload.

The 4.6 Hot fix still not works. It requires module UE4Editor-Kinect4Unreal.dll should be rebuilt.
Anyone who can work with UE 4.6 now?

Viewtron:
As we mentioned yesterday we have updated our introduction level to include a set up of the meta rig from blender in avateering.

4.6 hot fix with Introduction

If you are using your own custom skeleton we have left included our development tool, which is a rotator that increments 90 degrees upon certain button presses:

U: Increment Pitch
I: Increment Yaw
O: Increment Roll
P: Print current values

This rotator is not connected to anything currently, however it can be hooked up to individual bones in the anim blueprint and incremented until they are correctly oriented. You can then use these values and move onto the next bone. The blueprint itself contains commenting and should be easy to use.

Trungswat:
The most likely problem could be if you have not converted your project file to unreal 4.6. To correct this right click on your project file and choose change engine version to 4.6.

Another possibility that could cause this issue would be using a 32bit operating system. What sort of computer set up do you have and what is your operating system?

It’s unfortunately. It’s still not working. I made a new project by UE 4.6 then copy the plugin to Plugins folder of the project.
and I’m using x64 OS. Do you have any idea?

trungswat:
We have thus far been unable to reproduce your problem. We can fairly easily reproduce a problem that gets the same message by having a mismatched plugin and project versions, however this is always fixed by the previous suggestion. We obviously want this sorted out as soon as possible, it may help if you could send us information from your dxdiag and/or your broken project.

A second alternative that we may have found is the setting for the development environment in visual studio. We currently build our plugins with this set as development editor and a mismatch here may also cause this error.

The third thing that we have found that may have caused this error is building your engine version for github source code. The plugin was designed to operate on the binary version of the engine (downloaded directly from the launcher). This may also cause a mismatch like previous situations.

Thanks for the great support! Your tool was really helpful!