Kinect 4 Unreal

ZAxis · May 10, 2015, 6:12am

Thanks for your amazing plugin K4U.our team are working on an animation witch we will render it with unreal engine 4.for facial animation we use joint-based animation with Autodesk MatchMover and results are good enough .my Question is: Is it possible to use your plugin to completely implementing facial animation in UE with same quality instead of our approach that Autodesk MatchMover ?
Thanks.

anonymous_user_4e6bdf33 · May 14, 2015, 6:20am

p_cat:

If you want to determine when a user passes a certain distance using the depth sensor is probably overkill. Using the pitch adjusted joint position node you can break the output into its components and use the X vector as the distance the user is.

plockhart:

Thank you very much, donations are an interesting thing though. As we are a commercial entity who sell it just kind of feels dirty to also accept donations so we are going to have to turn you down. I suppose you could buy the example content if you haven’t already?

Viewtron:

To help with mirroring I would have to see your blueprints to see just where things are going wrong. And it is still currently not possible to package.

ZAxis:

All the work we have done thus far with facial animation has been morph target based, as the data that we receive from the Kinect is floats. That doesn’t mean that data can’t be then used in a bone based systems but it would take some setting up and we have not yet publicly released the face version of our plugin (although we could share it for good causes). The quality we get is very similar to the Kinect HD face demo from the SDK, however with access to smoothing options inside blueprints we can sanitize that data to a reasonable degree.

Techrocket9 · May 14, 2015, 6:34am

[FONT=Times New Roman]I hope this plugin gets updated with 4.8 support once the official release happens. I have some big ideas for using K4Unreal with 4.8’s Procedural Mesh Blueprints.

anonymous_user_f426e3f4 · May 21, 2015, 3:24am

there is an error when run packed project

i have see the FAQ , is that mean i can only run my project in the editor?

anonymous_user_4e6bdf33 · May 21, 2015, 4:40am

Unfortunately for the time being yes. Epic have said they have had plans for complete plugin support since 4.3 however they have not yet gotten around to implementing it.

RVillani · June 9, 2015, 5:50pm

Will I be able to use the plugin on 4.8 when the final version come out? I’m using it on a project with lots of foliage and the FPS is s**t. On 4.8 it’s better and it would help a lot to be able to update my project.

anonymous_user_4e6bdf33 · June 10, 2015, 1:25am

We are definitely going to be supporting version 4.8.

anonymous_user_1b674e2e · June 11, 2015, 1:31pm

Hi OpaqueMulimedia,

Quick Questions:

Does your plugin work with Kinect Xbox One + windows kinect adapter (I am assuming that it does but I just want to make sure before I purchase the kinect)?
Can I record character animations in Unreal via the Kinect Input to use during gameplay (i.e. not to control a pawn)? *Note: I am still familiarizing myself with U4 but want to see if this is conceptually possible as I get to learning more about how animations work

Thanks,

anonymous_user_4e6bdf33 · June 12, 2015, 3:50am

The kinect v2 with the xbox and the kinect for windows are identical, the windows adapter just came bundled with the kinect for windows.

Also yes you can record animations with the inbuilt record function of unreal using the kinect, so more good news there. They may need to be cleaned up a little depending exactly what you want to do with them though.

anonymous_user_4e6bdf33 · June 15, 2015, 7:26am

We now have a plugin live for Unreal Engine version 4.8.

RVillani · June 16, 2015, 12:03am

Thank you guys for releasing the updated plugin so quickly!

I’ve managed to migrate my project from 4.7.6 to 4.8.0 with the new plugin (after some trial and error). There was a class crashing the editor every time I tried to compile it so I re-made it from scratch, testing at every new node if it compiled ok.
After finishing to reproduce the class and successfully compiling it, I tried to Play in Editor. The Editor crashed with some KERNEL error and a phrase “Couldn’t spawn play”. After that, I can’t compile it anymore (same as the original class I copied from). The editor crashes when I try to compile it. The crash error begins with 4 lines of Kinect4Unreal, so I can’t debug it.

To make sure it wasn’t some version migration problem, I was able to reproduce the crash migrating the BP (and it’s related assets) to an empty project and trying to compile it there. Here’s the test project with the problematic BP. Could you guys at , please, have a look and tell me if there’s a workaround (for something I might be doing wrong) or if it’s, indeed, a bug? The problematic BP is KinectTreePlayerController.

anonymous_user_4e6bdf33 · June 17, 2015, 1:34am

We’ve gone over test project and it compiled and ran all perfectly, so we’re not sure what is causing your problem. We have migrated several of our own projects without issue.

RVillani · June 17, 2015, 3:14am

Thanks for testing.
Unfortunately, my problem persists. On test project I can Play in Editor but, after some tests, I found out I can only compile KinectTreePlayerController Blueprint if I delete ‘Get Tracked Bodies’ on ‘ControlBodyTrack’ function. With that node, it crashes the editor with the four UE4Editor_Kinect4Unreal lines on the crash message when I try to compile that BP.

What do you guys recommend as the right workflow for updating a 4.7.6 project with plugin? I could try it and see if it works for my project.

UPDATE: Managed to compile the BP and PIE. But I had to parent it to Actor, instead of PlayerController, and place an instance of it in the World (since it’s not a PlayerControler anymore).
My guess is it has something to do with the KinectPlayerController class being absent in this version. But I can’t be sure. Anyway, it’s unfortunate that I can’t place my tracking controls loop in the PlayerController anymore. It was more ‘Unreal MO’, I think.

anonymous_user_1b674e2e · July 15, 2015, 6:43pm

Has anyone else had trouble opening the Kinect 4 Unreal Intro package from the marketplace. After downloading it from the marketplace I try creating a project however there is no project file located in the directory that is created for the project. Is there something or some step I am missing.

I am using the Kinect with windows 8.1 and Unreal version 4.8.2

Thanks,

anonymous_user_4e6bdf33 · July 16, 2015, 3:43am

There was a problem during the updating of marketplace files that had several of the project files go walk about, we are in the process of fixing this as we speak. For the time being here is a direct link to our copy of the project files.

plangton · July 21, 2015, 2:17pm

Can we convert the depth data to camera space ( 3d position space ) using your plugin ?
I am interested to create a plane mesh from the depth data I capture from kinect and I want it to be identical so can be projected over the same object again

anonymous_user_ba17f0fc1 · July 24, 2015, 7:17pm

Hi , I wanted to ask if it’s possible with unreal engine to also show kinect camera feed. something like this image :
Thanks

turick · July 25, 2015, 7:17pm

Another question for the team. I am looking to build a VR project and want the connect to track hand position, which would control the player’s hand in the game. I don’t need anything fancy, just the ability for the player to hold their arm straight in front of themselves, move their hand left, right, up, down, and have the kinect translate that to the first person actor in the game. Smoothing for me would be very important. I like the avateering demo in the intro video, but just want to make sure it’s easy to rig up to the actual FPS pawn in my game.

BlindCuervo · July 26, 2015, 9:50am

Hello Guys,

get the following error when i try to compile my game:

1>EXEC : error : Couldn’t find module rules file for module ‘Kinect4Unreal’.

Unreal version 4.8.2 and plugin version 1.1c.

RVillani · July 27, 2015, 12:02pm

I’ve been working with this plugin for a while in two of my company’s projects and I think I should try to help here. Because many of my problems were solved by people who commented here before me.

@plangton
AFAIK, you can’t. BUT, using the depth camera texture, you could make a plane with a shader that outputs to the depth offset channel. If you’re trying to do what I think you are, that should solve.

@m0j1
Yes it is. The Kinect4Unreal plugin has functions for using the IR, Depth and Color camera feeds as textures updating in real time (and all at once, too).

@turick
The “arm straight in front” position is tough to work with because of Kinect’s imprecision when limbs are pointing at the camera. You WILL need to make some strong smoothing algorithm to compensate the jittering. Not complicated if you use the easing functions in Unreal, I believe. But you’ll have some lag. Better think of another posture or use the Leap Motion sensor instead of Kinect.

@Sijoun
Unreal is not compiling games with plugins with no source code for now. You can either run it in standalone window, play in editor in full screen, wait for Epic to solve this problem or write your own Kinect Plugin, thus having the source code and being able to compile. I’m almost doing it since I’ll really need to compile a project for a client. Also, search in the forum for another plugin. I remember seeing a guy doing his Kinect plugin, also. Maybe he provides the source code.