Kinect 4 Windows v2.0 Plugin

[=;107584]
Microsoft said they were looking into it, but i hope Epic/Microsoft can create an official plugin. So we can get more advanced features and integrations :slight_smile:
[/]

Just getting my Rift DK2, and been using UDK/UE for a few years now, unfortunately not a coder/scripter =P

This is really impressive stuff, amazed by it and looking forward to future developments!

Thank you you very much :slight_smile: I could make it work!

Thank you!

[=;103919]
There are a couple errors from assets that weren’t included (or broke/corrupted). Mainly these errors (which are ok)


Error /Game/DemoRoom/Materials/M_DemoWall_Inst : Can't find file for asset. /Game/ExampleContent/Audio/Physical_Materials/Rubber
Error /Game/DemoRoom/Materials/M_DemoWall_Inst_3 : Can't find file for asset. /Game/ExampleContent/Audio/Physical_Materials/Metal
Error /Game/DemoRoom/Materials/M_DemoRoomTiles_Inst : Can't find file for asset. /Game/ExampleContent/Audio/Physical_Materials/Concrete
Error /Game/DemoRoom/Materials/M_DemoWallPattern_Inst : Can't find file for asset. /Game/ExampleContent/Audio/Physical_Materials/Metal
Error /Game/DemoRoom/Materials/M_DemoWallPattern_NoText : Can't find file for asset. /Game/ExampleContent/Audio/Physical_Materials/Metal
Info Failed to load /Game/ExampleContent/Audio/Physical_Materials/Rubber.Rubber Referenced by M_DemoWall_Inst Property /Script/Engine.MaterialInstance:PhysMaterial
Info Failed to load /Game/ExampleContent/Audio/Physical_Materials/Metal.Metal Referenced by M_DemoWall_Inst_3 Property /Script/Engine.MaterialInstance:PhysMaterial
Info Failed to load /Game/ExampleContent/Audio/Physical_Materials/Concrete.Concrete Referenced by M_DemoRoomTiles_Inst Property /Script/Engine.MaterialInstance:PhysMaterial
Info Failed to load /Game/ExampleContent/Audio/Physical_Materials/Metal.Metal Referenced by M_DemoWallPattern_Inst Property /Script/Engine.MaterialInstance:PhysMaterial
Info Failed to load /Game/ExampleContent/Audio/Physical_Materials/Metal.Metal Referenced by M_DemoWallPattern_NoText Property /Script/Engine.MaterialInstance:PhysMaterial


But the fact you have this many errors means you probably don’t have the Kinect Plugin enabled (Window -> Plugins, Enable Kinect)
[/]

Hello,

Thanks for the hard work !
I was able to download the code and recompile with 4.4 without any issues.
I was also able to load the sample and let the auto-conversion do its job and everything works.

I just have two questions :
-is it possible to change it as a game plugin instead of a engine plugin ? I guess it needs access to lots of elements to the engine
-once I’ve recompiled, I cannot seem to distribute it to someone else who has also 4.4. It only works with my custom version of the engine. The editor is trying to recompile it even if it’s already done.

[=kt;107571]
Hi Lion,

Thank you for all your great work. Any plans on adding Kinect voice commands to your plugin?

Kevin
[/]

Already working on that :slight_smile:

[=BZysberg;121510]
Hello,

Thanks for the hard work !
I was able to download the code and recompile with 4.4 without any issues.
I was also able to load the sample and let the auto-conversion do its job and everything works.

I just have two questions :
-is it possible to change it as a game plugin instead of a engine plugin ? I guess it needs access to lots of elements to the engine
-once I’ve recompiled, I cannot seem to distribute it to someone else who has also 4.4. It only works with my custom version of the engine. The editor is trying to recompile it even if it’s already done.
[/]

Epic are aware of that issue.

[=;123077]

Epic are aware of that issue.
[/]

Yeah a coworker told me Epic spoke about how broken the plugin system is right now.
I can wait.

As for the freeze issue, I think I know why.
Most people choose to do all the updates for Kinect in a block of code. You chose to do it in a separate thread.
It’s pretty smart but something goes wrong. This lock happens :

FScopeLock lock(&mBodyCriticalSection); (line 349 of KinectSensor.cpp)

and it never unlocks therefore the freeze. It happens when you suddenly switch how many bodies are tracked (2 to 1, 1 to 0…).
It seems like the Kinect thread manages to goes twice inside this lock section before doing the GetBodyFrame.

Congrats on your Kickstarter !

Never mind. The freeze was coming from something else.
Just remove/comment the line 122 in KinectFunctionLibrary.cpp

WorldQuat *= KinectBones[ParentBone].Orientation;

Not only it removes the freeze but it also make the thing more accurate.

[=BZysberg;123707]
Never mind. The freeze was coming from something else.
Just remove/comment the line 122 in KinectFunctionLibrary.cpp

WorldQuat *= KinectBones[ParentBone].Orientation;

Not only it removes the freeze but it also make the thing more accurate.
[/]

I never encountered that issue, please send me a code to reproduce it.

Hey Guys!

this is a small video of the plugin in action.

https://.com/watch?v=9FGTWp2QUoU

I am uploading the demo so you could try it for yourself.

haha very nice lion, what kind of game was he playing?

It may have been from my code drop, as some people were having problems with the latest version you had (missing libraries/errors related to OpenCV and the OneEuro filtering). Just reverted the last 1-2 commits and updated some deprecated API’s, to get it properly compiling again.

The upload of the demo is almost done, so you will be able to check it out for yourself soon :slight_smile:

I am overhauling the whole plugin.

I have introduced a kinect body manager that keeps track if skeletons, notifies if new skeleton is detected so you will be able to assign it to a player using your own gesture recognition system.
also it tracks if an assigned tracking is lost skeleton is lost.
I am also thinking about deprecating the IkinectBodyFramelistenerInterface in favor of more robust Delegate based system.

Good thing i have a DK2/kinect aswell so i can actually try it out in VR haha.

The Skeleton tracking/losing is a really nice feature, and can’t wait to try it. Let me know if you need help testing it.

I have just pushed the 4.4.1 version, high chance that it will break your code/invade north korea

Well it didn’t take long for it to start World War 3 :-P, were there any engine edits or is it still 100% plugin?

Still 100% plugin and it is my personal goal to keep it that way.

You have changed quite a bit :-p, might have to see how you re implemented some stuff, time to fiddle around :slight_smile:

Enjoy!

If you have any questions PM me.

Hopefully if I’ll be able to get out of the office at a reasonable hour tomorrow I’ll start working on documentation for everything.

[=;123955]
I never encountered that issue, please send me a code to reproduce it.
[/]

It’s the same code as this :
https://drive.google.com/file/d/0B6pIAtEjvJCQcjgxcjZOdFljUm8/edit?usp=sharing
(posted above by )

It’s difficult to reproduce it just by yourself. I needed two people to do it. They both have to jump back and forward in front of Kinect.
At some point this line of code will try to access/modifiy a skeleton that’s not there anymore.

I also made a Blueprint that detects the id of the skeleton the first time someone is tracked and only tracks this id.
Then when nobody is tracked the id is reset.

http://i.imgur.com/1dJn3ZK.png

Can’t wait to try your new version !

I’ll look into it.

Meanwhile here’s the new demo :slight_smile:

Thanks !

A few questions :
1)There is no more ā€œEditorā€ folder. Did you merge the code or is it still needed ?
2)Are you going to update the Avatar demo ?
3)Are you going to provide the source for your new demo ?

[=BZysberg;126001]
Thanks !

A few questions :
1)There is no more ā€œEditorā€ folder. Did you merge the code or is it still needed ?
2)Are you going to update the Avatar demo ?
3)Are you going to provide the source for your new demo ?
[/]

  1. I saw that it was useless for avateering so it was removed.
  2. I will when the new body tracking system will be completed.
  3. I hope that I will have sometime this weekend to do that.