šŸ”„ [UE-Store] Real-time Full Body tracking for UE5 [Nuitrack][PLUGIN]

Hello

  1. Choose 90 or 270 in ā€œDepthProviderā€: ā€œRotateAngleā€ (C:\Program Files\Nuitrack\nuitrack\nuitrack\data\nuitrack.config)
  2. Try to choose ā€œWaistā€ instead of ā€œNoneā€ in the T-shirt BP
    image
1 Like

@Nuitrack, thank you kindly!
I managed to import some new meshes based on the avatar you have provided in the sample code of the plugin and it worked just fine. You might want to apply the ā€œwaist instead of noneā€ fix to the sample code, though.

1 Like

Has anyone successfully built a project using 5.4 and the latest plugIn? I am getting DLL problems so I followed the intructions about DLL placement. it says next to the .exe file in build, theres no .exe there but it did make the popup error go away but it still wont build.

i followed instructions to Tomatow question that involved modifying build.cs , i thred that but got same result, a failed build with no errors in the log.

any help appreciated

hello,When I run in the editor, the viewport turns white after about two or three seconds. Is this because I don’t have a license?


Hello. Yes. For the Unreal plugin to work, you need an AI Online subscription license. Perpetual and Trial are not supported in UE

Hi @Nuitrack. It is unclear to me, if the Unreal API needs a background app running for capturing?

Hello @Luci404
You need to install this https://github.com/3DiVi/nuitrack-sdk/blob/v0.37.24/Platforms/nuitrack-windows-x64.exe and activate the sensor there, but there is no need to keep it running.

In short:

  1. Install Nuitrack. Activate the sensor in the Nuitrack Activation Tool (you won’t need it anymore)
  2. Install the plugin. And then you work only in Unreal.
  3. Run the samples from the plugin folder, check how it works (If something doesn’t work, don’t forget to read the logs)

Hello @Nuitrack great plugin, i wanted to ask if the Perpetual license works with Unrealengine5 ?

Hello @IRANpower

1 Like

Does this mean that it also does not work with the ā€œNuitrack AI Perpetualā€ license? In general, you licenses seem quite confusing. Am I getting updates with the perpetual license, can it be used offline, etc?

Hello @Luci404
Yes, it’s really quite confusing right now
The plugin for UE works only with AI Subscription (Perpetual and Trial will not work). It will also not work offline.

A new update for the plugin has been released (Recommended version of Nuitrack Release Nuitrack v0.38.2 Ā· 3DiVi/nuitrack-sdk Ā· GitHub)

  • Unreal 5.5 support :bangbang:
  • Added the Metahuman blueprint. Now you can use your metahuman in a few simple steps and animate it in Real-time using Nuitrack :man_running:
  • Updated Avatar Sample for using Manny Avatar (The old Mannequin example that does not use Epic Skeleton also remains BP_AvatarMannequinBlueprint) :robot:
  • Experimental Facetracking based Head rotation :no_mouth:

(post deleted by author)

Hello @Nuitrack
I’m failed to open the fitting room sample in 5.4 and 5.5, but It works in 5.2. I can’t figure out the reason.
In 5.2, The cloth and pants seems not fit to the player body. It looks like the 3d body is closer to the camera than real situation. Please see the attachment.

My device:

  1. orbbec femto bolt
    2.Nuitrack version 0.38.2. Also toggle on ā€œUse Nuitrack AIā€

Hello @beatless :waving_hand:

  1. Because a depth map doesn’t accurately match an RGB image, therefore, they should be aligned. To turn on depth-to-color registration, open nuitrack.config from the folder C:\Program Files\Nuitrack\nuitrack\nuitrack\data and set DepthProvider.Depth2ColorRegistration to true

  2. Go to SceneCaptureComponent2D on BP_ClothAvatar and set fov the same as in your sensor

  3. Change resolution like your color frame resolution RT_ClothRenderTarget

We are currently actively working on improving this sample so that everything works automatically.