Pose tracker plugin for Unreal Engine

Community Showcase
,
1

Hi Everyone!

I’ve made a Pose tracker plugin for Unreal Engine based on the open source MediaPipe framework. It employs machine learning to infer up to 33 3D full-body landmarks from RGB frames. Works with any webcam, no special hardware is required.

KEY FEATURES:

  • Full body pose tracking
  • Works with any webcam, no special hardware is required
  • 2D, and 3D pose landmarks
  • Lightweight solution
  • Indoor / Outdoor
  • Touchless interactions
  • 100% blueprintable no coding knowledge required
  • Supported platforms: Windows 64bit
  • Compatible with Unreal Engine v4.26, v4.27, 5.0EA

Available on: http://pose.uplugins.com

2 Likes
2

Hey @AdamHorvath,

The hard work definitely paid off on this one. This seems like such a genuinely incredible tool. Forgive me for my lack of knowledge in this respect but could you potentially create idle animations with this plugin?

1 Like
3

can be the camera change to be use with a phone?
iphone and android, or just a webcam?

cheers!

4

It is for windows only with webcam. Can’t be used on mobile devices.

5

Hello!

I’ve released an updated version of the plugin for Unreal 4.26/4.27 and 5.0.1

Pose tracker plugin V1.1

Updates:

  • FMediaPipeConnector class renamed to FMediaPipePoseConnector to prevent conflicts
  • Fixed plugin loading stucks at 75% (UE5)
  • UI fixes (UE5)
  • Minor bugfixes

Get it from http://pose.uplugins.com
More plugins: http://www.uplugins.com

6

Hi, I just tried your demo from here: Pose tracker plugin for Unreal Engine - POSE TRACKER

Unfortunately the demo failed to activate my web camera, and crashed when I clicked “Stop”. As far as I can tell from settings, the app never requested access to my Web Camera.

Setup:

OS: Windows 11 Pro (10.0.26100 Build 26100)

Processor: AMD Ryzen Threadripper 7970X 32-Cores

Memory: 128GB

GPU 0: GeForce RTX 5090

GPU1: GeForce RTX 4090

==========

If it would be helpful to you, I would be happy to test demo builds or provide a crash report.

7

UPDATE: Using a different web cam solved the problem.

The crash occurred when I was using a Logitech Brio web camera.

Switching to Elgato FaceCam MK.2 everything worked without issue.

In Windows Device Manager, the Logitech camera shows up under “Imaging devices” while the Elgato camera shows up under “Cameras” although each can be selected in your app when connected.

In testing another application that used OpenCV for Python, I had the same problem but in that case noticed the following console error: global cap_msmf.cpp:1795 CvCapture_MSMF::grabFrame videoio(MSMF): can’t grab frame. Error: -2147483638

If there is anything further that I can do to be helpful please let me know in a reply.