🔥 [UE-Store] Real-time Full Body tracking for UE5 [Nuitrack][PLUGIN]

Development Programming & Scripting
, , , ,
1

Hi there,

Nuitrack_ue_plugin169

:arrow_right: Download plugin for free:
https://www.unrealengine.com/marketplace/en-US/product/nuitrack-real-time-body-tracking

We’re happy to present our real-time body tracking for Unreal Engine 5. It could be used without writing any code, simply connecting Nodes in Blueprints :computer:
It’s perfect for developing games :video_game:, fitness apps :basketball_man: and other interactive projects with body tracking.

Get started with examples showing different Nuitrack capabilities:

  • SkeletonJoints - check what the camera sees and how the skeleton joints are tracked
  • AvatarMannequin - animate a game character using Nuitrack skeleton :man_dancing:
  • FittingRoom - how to create a virtual fitting room :dancer:
  • GesturesDemo - demonstrates how the gesture recognition works :wave:
  • HandTracker - how to use hand tracking to interact with the UI :raised_hand::raised_back_of_hand:

If you’re interested in Skeleton Tracking :man_dancing: with Orbbec, Intel RealSense or Microsoft Xbox Kinect sensors (v1, v2, Azure) :movie_camera:, you’re welcome to try https://www.unrealengine.com/marketplace/en-US/product/nuitrack-real-time-body-tracking

Tutorials :mortar_board: :
:rocket: How To Start - https://forums.unrealengine.com/t/real-time-full-body-tracking-for-ue5-nuitrack-plugin/1331349/9
:skull: How To Visualize Skeleton - https://forums.unrealengine.com/t/real-time-full-body-tracking-for-ue5-nuitrack-plugin/1331349/18
:dancer: How to Animate Avatar (Text) - https://forums.unrealengine.com/t/update-real-time-full-body-tracking-for-ue5-nuitrack-plugin/1331349/24?u=nuitrack

Troubleshooting:

Plugin Rebuilding

How to Rebuild the Plugin
No rebuilding is required for version 5.2.
For versions above, Unreal requests rebuilding. In this case, three options are available: Automatic rebuilding (recommended), using pre-builded libraries, and manual rebuilding.

Automatic Rebuilding:

  1. Start opening your project. When the message “Missing … Modules”, click the “Yes” button. The window will close and after some time your project should open automatically.
  2. If an error occurs, try restarting the PC and repeat step 1.
  3. If nothing helps, try other rebuilding options

Using pre-builded libraries:

  1. Open the nuitrack plugin folder ({Your project}\Plugins\NuitrackPlugin\Binaries\Win64)
  2. Pay attention to zip archives. If there is a UE version you need among them, then unzip them into this folder with the file replacement
  3. Try to open the project

Manual Rebuilding:

  1. Create a C++ project in Unreal Engine
  2. In the menu bar, select “Tools/Open Visual Studio”
  3. Copy the Nuitrack plugin to the Plugins folder in your project (according to the instructions)
  4. In the Visual Studio menu bar, select “Build/Build Solution”
  5. If an error appears, close the Unreal Editor (in the task manager, make sure that the process is not running there “UnrealEditor.exe”) and repeat the previous step. You can check UE tutorial Building Plugins | Community tutorial
  6. If the building was successful, you can open the UE project.
7 Likes
2

You can also ask questions in our community and learn more about Nuitrack on github

Nuitrack Community https://community.nuitrack.com/
Github https://github.com/3DiVi/nuitrack-sdk

2 Likes
3

Nuitrack has two skeleton tracking modes “Classical” and AI.
The “Classical” mode has a high speed of operation and can be run even on mobile devices:

2 Likes
4

AI tracking uses the latest machine learning technologies :robot:, which allows you to better recognize complex poses. More sensitive to device performance than the “classical” algorithm
(Can also recognize some objects)

3 Likes
5

At the moment, some NUITRACK functionality is not implemented in the plugin:

  • face tracking (bbox, emotions, gender, age, years) :smiley: :rage: :neutral_face: :astonished: [ :baby: :older_man:]
  • object recognition :handbag: :school_satchel: :iphone:
  • multisensor tracking :movie_camera: :video_camera: :camera_flash: :camera:

So far, this can only be used through C++ code. Write to us if someone needs this functionality in the form of blueprint nodes

2 Likes
6

:arrow_up:

At the moment there are several blueprints and levels for self-study. If the UE community has an interest in our plugin, we will definitely add tutorials and videos. :nerd_face:

1 Like
7

it is totally insane… i can’t wait…!! please give me a nuitrack tutorial for ue5!!!

2 Likes
8

This week we plan to start publishing small video tutorials, subscribe to our channel :bangbang:

  • How to start
  • Blueprints and Levels Overview
  • How to Visualize Skeleton
  • How Animate Avatar (3D Model\Skeletal Mesh)
  • …
2 Likes
9

Nuitrack And Unreal. How to start :tada:
Don’t forget to subscribe to the channel and like this video :wink:

Coming soon: tutorial “How to Visualize Skeleton”

10

:muscle:

image
image1178Ă—388 40.8 KB

11

Hey guys.
Great plugin. Sounds like something I can use for my project.

I have a couple of questions.

  1. Can i use multiple Kinect sensors using this plugin? So technically I have 3 kinects connected to my system, and I want tracking data of closest/any tracked person from all 3 sensors. Possible?

  2. Does this plugin support tracking via LIDAR? Can I track player movements / location via LIIDAR?

Blueprints are preferred but not a necessity, I just need these functionalities and I am perfectly fine with using C++ for these.

(Using this forum instead of your community to drive communication here)

1 Like
12

Hello @m_alitanveer

  1. Nuitrack has support for working with multiple sensors at the same time. There are no necessary nodes in blueprint yet, but you can use it via C++. This example should help you https://github.com/3DiVi/nuitrack-sdk/blob/master/Examples/nuitrack_multisensor_sample/src/main.cpp#L67
    By the way, which kinects do you want to use (v1, v2, azure)?
  2. Which sensor do you want to use? RealSense LiDAR L515 is working.
1 Like
13

Could you make a tutorial how to use two kinect v2 to get a better tracking?

1 Like
14

Would love to see the next steps please :slight_smile:

15

At the moment, this feature is under active development.
Stay tuned: https://github.com/3DiVi/nuitrack-sdk

  • :soon: Holistic skeletal tracking with multiple sensors
16

@Nuitrack

Cheers.

I have used your plugin in my project. It work correctly in my editor. But if I would like to package my project I can’t. First I got plugin modul error. I solved that problem, but now I absolutely cannot package the project. I got unknown error. Without the plugin I can. What do you think what could be the problem?

Thanks for you reply.

1 Like
17

Hello @T0maytoe

Can you tell me which version of Unreal you are using and provide any logs or screenshots?

This information may also help you
https://forums.unrealengine.com/t/real-time-full-body-tracking-for-ue5-nuitrack-plugin/1331349/20?u=nuitrack

18

Hello. And here is a new video tutorial that will show how to work with joints and visualize the skeleton :skull_and_crossbones:

19

image
image1600Ă—900 52.2 KB

Each sensor has its advantages and disadvantages. Be sure to study the manufacturer’s documentation.
And you can study the general recommendations in this article
https://github.com/3DiVi/nuitrack-sdk/blob/master/doc/General_preparations.md

20

It doesn’t seem to be relevant anymore. Now no additional actions are required to build

Deprecated Build Instruction

How to Build:

  1. Open {YourProject}\Source\{YourProject}\{YourProject}.Build.cs
  2. Add PrivateDependencyModuleNames.AddRange(new string[] { "NuitrackModule" });
  3. “Platforms”->“Windows (for example)”->“Package Project”

image
image947Ă—205 32.6 KB

:bangbang: There is one problem with the build now. The compiler complains about Try\Catch in the file NuitrackUtils.cpp in Update(). You can either delete this block yourself and then recompile it, or wait for the next plugin update.

image

1 Like