I’ve finished other parts of the project, now it’s time to look at this again.
I’ve been using DeepLabCut to generate 2D markers on animal bones from just videos.
Challenge is that it doesn’t actually seem to do any 3D skeletal pose estimation unless you set up multiple cameras to triangulate bone positions. Now that is 2 steps forward and 5 steps back for motion capture.
What I am hoping for is a program that knows a skeleton itself provides a lot of the neccessary information to turn a 2D video from a single camera into a 3D pose. Skeletal structure, bone angle constraints, bone lengths etc. But what program does this? What can I feed a video of a mouse to extract estimated 3D skeletal poses from?
Say I have the skeleton. every bone has a transform, length, and it’s known that twisting one beyond a certain angle is not healthy. Part of the skeleton is not visible on the single cam video because it is behind the body or behind an object. But so what? You can fill in the blanks on that data. Am I missing something? The skeletal structure and bone characteristics are known. Blanks can be filled with inverse kinematics or directly from the training data.
Just what program does it? 