Announcement

Collapse
No announcement yet.

Neo Kinect - easy access to the Kinect v2 capabilities in your games

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

    [PLUGIN] Neo Kinect - easy access to the Kinect v2 capabilities in your games

    With the Neo Kinect plugin you can use the Kinect v2 sensor advanced capabilities within Unreal Engine, with easy to use Blueprint nodes, all nicely commented, or directly through the C++ methods. Take a look at the Quick Start Guide to see how it works and have an idea of what you can achieve with it!

    Click image for larger version  Name:	Avateering.jpg Views:	1 Size:	255.7 KB ID:	1429302Click image for larger version  Name:	ContentExamples.jpg Views:	1 Size:	271.5 KB ID:	1429303
    Click image for larger version  Name:	BlueprintNodes.jpg Views:	1 Size:	445.7 KB ID:	1429304Click image for larger version  Name:	Frames.jpg Views:	1 Size:	246.0 KB ID:	1429306
    Click image for larger version  Name:	FaceTracking.jpg Views:	2 Size:	324.1 KB ID:	1429309Click image for larger version  Name:	Remapping.jpg Views:	1 Size:	295.4 KB ID:	1429307Click image for larger version

Name:	WidgetTextures.jpg
Views:	1684
Size:	237.3 KB
ID:	1429337Click image for larger version  Name:	ProductionProven.jpg Views:	1 Size:	449.8 KB ID:	1429308

    Robust and fast

    The plugin was created with performance and usability in mind, so you can track all 6 possible users, their faces and enable all of the Kinect's frame types (color, depth, infrared etc) at the same time with almost no hit in performance at all. Sensor polling is made in its own thread and a custom texture type was created just for the high-res-realtime updates, which is still compatible with the material editor system. If you need to, there are even functions to access the textures pixel values.
    No need for components

    The sensor is unique, no matter how many Actors or Widgets are using it. So, instead of needing to add components or extend specific Blueprints, you just call functions like with a function library. That way you can control the device from any Blueprint, including Widgets.
    Advanced Remapping

    Besides access to the standard Microsoft Kinect API coordinate remapping methods, the plugin also comes with other remapping features that facilitate AR applications, like getting the location of a joint in the Color frame without losing its depth information. Every location and orientation was adapted to Unreal's coordinate system and Joints transforms are compatible with the Engine's Mannequin character rig.
    Fully production proven

    I've used Neo Kinect a lot (more than a year) before releasing to the public and removed all bugs found so far, besides making a lot of performance improvements. It was used in applications that go through a whole day without crashing and packages without problems.
    Technical Details

    Body tracking:
    • Tracking of up to 6 simultaneous users's skeletons, with 25 joints each
    • Users leaning angle, tracking confidence, Body edge clipping, hands states
    • Per Body found/lost events
    Face tracking:
    • Location and orientation of up to 6 simultaneous users's faces
    • Face points (left and right eyes, nose and left and right mouth corners) in 3D and 2D (Color and Infrared space)
    • Faces bounding boxes in Color and Infrared frames space
    • Expressions reading (Engaged, Happy, Looking Away, Mouth Moved, Mouth Open and Left and Right Eyes Open/Closed) and if users are Wearing glasses or not
    • Per Face found/lost events
    Sensor control:
    • Global bodies/faces tracking events (found/lost)
    • Init/Uninit sensor
    • Get sensor tilt, ground plane normal and sensor height
    Remapping:
    • 3D camera location to Color texture (optionally with depth) and to Depth texture
    • Find depth of a Color texture location
    • Depth point to Color point and to 3D location
    Frames (textures):
    • Get each frame FOV and dimensions
    • Toggle frames usage individually
    • Sample a pixel value from the Depth frame and find the depth of a Color pixel

    Network Replicated: No
    Platform: Win64 only

    Example Project: NeoKinectExamples.zip
    Quick Start guide: NeoKinect-QuickStart.pdf
    Attached Files
    Gain easy access to the Kinect v2 capabilities in your games
    Last edited by RVillani; 02-15-2018, 05:03 PM.
    Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
    Unreal products: Dynamic Picture Frames, Neo Kinect

    #2
    How to use kinect? Can animations be made for the game?


    Some more detail please...

    Comment


      #3
      Originally posted by ShaNexxx View Post
      How to use kinect? Can animations be made for the game?


      Some more detail please...
      This plugin is to use Kinect in game. As a controller, for example.

      I've used it to make games the player can use just his body to control it and play and also a virtual dresser. If you'd like to use it to make animations, you can record the skeletal data during game in a file and then load it to play the animations.
      Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
      Unreal products: Dynamic Picture Frames, Neo Kinect

      Comment


        #4
        Hello, I'm about to buy it. IS there any option to try build or at least see the video? I would like to know what's the delay and how accurate it tracks.

        Comment


          #5
          Originally posted by VR VISIO GROUP View Post
          Hello, I'm about to buy it. IS there any option to try build or at least see the video? I would like to know what's the delay and how accurate it tracks.
          Hi, sorry for the delay. I'm in a small room until a week from now, so it's bad for recording Kinect videos. I'll do it and add to the market page asap.

          In the meantime, if you want to, we can chat on Skype and I can show it to you. Also, there's an app I made with it that you can download the demo and test it. But I can only send the link on private, so send me a message on Skype: r_skull
          Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
          Unreal products: Dynamic Picture Frames, Neo Kinect

          Comment


            #6
            Thank you for the reply. I will contact you on Skype then

            Comment


              #7
              I download the exanple , and followed the guid ,but when I run the example ,it crashed. I have bought the plugin ,when I created the my new project(C++) , it run well..... So , the example project is limited? my Eninge version is 4.18.

              Comment


                #8
                Originally posted by gaofei7439 View Post
                I download the exanple , and followed the guid ,but when I run the example ,it crashed. I have bought the plugin ,when I created the my new project(C++) , it run well..... So , the example project is limited? my Eninge version is 4.18.
                Mine is also 4.18. It crashed before or after you installed the plugin? What were you doing when it crashed? Can you make it crash again and tell me how to reproduce it?
                Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
                Unreal products: Dynamic Picture Frames, Neo Kinect

                Comment


                  #9
                  4.19 UPDATE

                  I've already submitted a plugin files update for the Marketplace to enable you to add the plugin to UE 4.19 version, but it can take a while for Epic to finish the new build and update it for you.
                  So please, let's be patient
                  Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
                  Unreal products: Dynamic Picture Frames, Neo Kinect

                  Comment


                    #10
                    Originally posted by RVillani View Post

                    Mine is also 4.18. It crashed before or after you installed the plugin? What were you doing when it crashed? Can you make it crash again and tell me how to reproduce it?
                    1, After I installed the plugin, when i run the example ,the example crashed.
                    2, I met a new trouble. I have bought the plugin and installed it on version 4.18. But now I also want to install the plugin on Version 4.16.3 . From Epic Luncher I canot installed . also
                    I copyed the "NeoKinectUnreal"file form D:\Program Files\Epic Games\UE_4.18\Engine\Plugins\Marketplace , to Version 4.16 ,it did not worked

                    Comment


                      #11
                      Originally posted by gaofei7439 View Post

                      1, After I installed the plugin, when i run the example ,the example crashed.
                      2, I met a new trouble. I have bought the plugin and installed it on version 4.18. But now I also want to install the plugin on Version 4.16.3 . From Epic Luncher I canot installed . also
                      I copyed the "NeoKinectUnreal"file form D:\Program Files\Epic Games\UE_4.18\Engine\Plugins\Marketplace , to Version 4.16 ,it did not worked
                      1. Are you still facing that crash? Like, does it happen every time you do something specific?

                      2. The plugin is programmed for 4.18 (and 4.19 as soon as Epic approves my update submission), and it won't compile on 4.16 because of:
                      • changes to the C++ Soft Object Reference types.
                      • the .uplugin file sets the target version to 4.18, so 4.16 will refuse to use it.
                      • the custom texture type (DynamcTexture2D) uses code very specific to each version. I can rarely update the plugin without having to re-write it.
                      I'd like to say something like "if you really need to use it on 4.16, call me on Skype and I'll help you" but it would require a LOT of changes for that version and a LOT of time. More than it would take you to upgrade your project to 4.18.
                      Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
                      Unreal products: Dynamic Picture Frames, Neo Kinect

                      Comment


                        #12
                        Originally posted by RVillani View Post
                        4.19 UPDATE

                        I've already submitted a plugin files update for the Marketplace to enable you to add the plugin to UE 4.19 version, but it can take a while for Epic to finish the new build and update it for you.
                        So please, let's be patient
                        Epic has finished building the new binaries and you can now use Neo Kinect in UE 4.19
                        Freelancer Game Dev Generalist and Unreal Consultant | Portfolio
                        Unreal products: Dynamic Picture Frames, Neo Kinect

                        Comment


                          #13
                          Hello,

                          I just purchased the plugin. I'm trying out the example project. It works great in editor When I try to publish out the application it compiles successfully.

                          When launched it starts up black so I set the Game Default Map to the Examples. When I try to launch that I get "The UE4-NeoNinectExamples Game has crashed and will ..."

                          Any suggestions?

                          I am using Engine 4.18.3 and it looks like the example project is 4.18

                          Spencer

                          Comment


                            #14
                            You need to read the README in the example project, follow the instructions there and copy the sdk files to the project for it to work. the same files will also need to be added to the build after you packaged it under WindowsNoEditor\*your_project*\Binaries\Win64\

                            Comment


                              #15
                              DarkAnimator Thanks for getting back to me on that.

                              I have used Neo Kinect and published it in my own project now successfully. So for the record, great plugin.

                              Comment

                              Working...
                              X