šŸ”„ [UE-Store] Real-time Full Body tracking for UE5 [Nuitrack][PLUGIN]

Development Programming & Scripting
, , , ,
21

@Nuitrack

Hello!

Thank you very much now works for me. Your post was helpful.

1 Like
22

Mail

:bangbang:A new version of the beta-plugin has been released:

  • Simplified Game Building. Now you donā€™t need to edit any files, just do the build.
  • Removed errors from the output log that appeared at startup
  • The size of the plugin has been significantly reduced

:firecracker: New Version Beta Nuitrack UE5 plugin šŸ”„ [Update] Real-time Full Body tracking for UE5 [Nuitrack][PLUGIN]

I also remind you that tutorials have appeared on our channel (Subscribe Nuitrack Channel so as not to miss):

  • How To Start
  • Skeleton Visualization
  • and more (soon)ā€¦

1 Like
23

:tada: A tutorial on how to make an avatar animation will be published soon on our community :wink:

24

:man_dancing: How To Make Avatar Text Tutorial :dancer:
Mail

:tshirt: The tutorial will show you how to animate an avatar in the Unreal Engine using Nuitrack.

To Top

25

Hello @Nuitrack,
the plugin looks great for motion tracking. :+1:
I was wondering if it also can be used for depth tracking?
The use case would be for a top-down camera tracking objects and people.

1 Like
26

Hello @lkav
Unfortunately, working with a top-down view is not supported right now. It is recommended to install the sensor exactly in front of the monitored objects or at a slight angle

27

:tada:
The plugin has been updated (stability improvements)

:bangbang: Now UE5 does not crash after the end of the trial period and in some other cases.

28

:rocket: We will publish the plugin in the UE-store soon

There will also be a newer version with an improved avatar. :fire:

29

:fire: Good news everyone :fire:
Now you can download the plugin from the store!
Download, try, leave comments and suggestions.

  1. Install Nuitrack nuitrack-sdk/Platforms at master Ā· 3DiVi/nuitrack-sdk Ā· GitHub
  2. Install Nuitrack Plugin via Epic Games Launcher
  3. Go to ā€œPluginsā€ and select ā€œNutrackā€
  4. Restart
  5. Play Sample Levels
  6. ???
  7. Profit

:arrow_right: In the latest update:

  1. The error output has been improved, they have become more understandable
  2. Now the avatar can be rotated in any direction
1 Like
30

It would be appreciated if you add support for hand tracking and gestures on Android phones. :slightly_smiling_face:

1 Like
31

:bangbang: Help us decide which way to develop the plugin. Please fill out this :arrow_right: short form. :bangbang:

33

Hi! Iā€™m trying to use Nuitrack to track my handā€™s position using Kinect v2 in Unreal.

I tried following your tutorials but it seems unreal doesnā€™t receive Kinectā€™s signal. Do I need to do any setup? Am I missing any software?

I have used TouchDesigner to do the tracking before and it works but Iā€™m looking for a plugin solution right now.

How can I verifiy that Nuitrack is receiving the video data?

Thank you.

34

Hello @VAG.3D

It looks like you donā€™t have Nuitrack Middleware installed (touchdesigner is a special case)
Please check out this ā€œHow To Startā€ tutorial

1 Like
35

Hello everyone, I bought a nuitrack AI license to be able to track a rotating skeleton, in the SDK window everything happens ok, you can see that the skeleton rotates and is tracked stably, but in Unreal everything remains as it was in the demo version, the skeleton is tracked but does not rotate around z axisā€¦ I canā€™t understand what to do with it, maybe someone has a similar story, please tell me what to do?

36

Hello
To enable AI mode, you need to edit the ā€œnuitrack.configā€ file according to this instruction: nuitrack-sdk/doc/Nuitrack_AI.md at master Ā· 3DiVi/nuitrack-sdk Ā· GitHub

If the joints do not behave stably in AI mode in UE, try the previous version of nuitrack (For example nuitrack-sdk/Platforms/nuitrack-windows-x64.exe at v0.36.17 Ā· 3DiVi/nuitrack-sdk Ā· GitHub )

1 Like
37

Trying this in 5.2 with the marketplace version of the plugin and an Orbbec Femto Bolt ā€¦ trying to PIE any of the samples throws the error in this screenshot:

nuitrack_error
nuitrack_error708Ɨ436 41.5 KB

ā€¦ any idea what Iā€™m doing wrong?

38

Hello @Noah.Norman
We are already working on a solution to this problem. Try using the previous version of Nuitrack (0.37.12)
https://github.com/3DiVi/nuitrack-sdk/blob/v0.37.12/Platforms/nuitrack-windows-x64.exe

39

I was able to get these examples to work with 0.37.12 thanks @Nuitrack. Do you have any intention of improving the Avatar example implementation to a more sophisticated puppeting approach?

1 Like
40

@Noah.Norman :tada:
We plan to gradually improve the examples with avatars. For example, simplify setting up custom avatars, ā€œsmooth movementsā€ and add a Metahuman example.
If you have any suggestions or comments, you can write them and we will try to take them into account in the future.

1 Like
41

Iā€™d love to be more helpful but I donā€™t know a ton about animation. With that said, Iā€™m pretty sure just transforming every joint is not the optimal approach to avoid the avatar mesh tearing and getting balled up, etc ā€¦ I think you want something that uses IK / bones but again Iā€™m a little out of my depth here.

Might be useful to refer to the Zed UE implementation (MIT licensed):

1 Like