I have a pretty in the weeds question about a VR scene I’m making. I made a plugin to enable native FBT support through steam VR. And the FBT plugin is largely completed and functional. However I am having some serious problems getting the FBT’s to actually move the control rig correctly. The control rig is being controlled by captured poses from the composed locations of the skeletal mesh sockets and the Motion Controllers world transform and then being made relative to character mesh. I am having a lot of issues with basically the entire pipeline. Any insight is greatly appreciated.
This repo should be helpful,
Yknow that’s actually pretty useful but not really what I was looking for. I appreciate it, but The benefit of my plugin over this one is ease of use. And it compatible with all trackers. the motion trackers aren’t the issue. its the control rig. My tracker components are 100% accurate and are fully compatible with BP & C++. So while this was interesting its not really helpful to my current problems. The mocap aspect is iteresting too, but not nearly as intuitive as well as useful as my own. The root problem is less of a FBT tracker not working issue, and more of a the trackers are moving completely correctly in game and follow the SteamVR overlay with 0 slop. The issue is likely the differences in rotational basis’s between between the trackers and the control rig. To be clear the issue is not the tracker plugin. It is the relationship between the skeletal mesh, the sockets, the control rig, and the motion controllers. Not the motion controllers themselves.
I made a plugin using Vive Trackers + Control Rig and I’m about to upgrade to use FBIK node rather than separate nodes.
Consider that there’s a bif of transform conversion that goes between trackers and Control Rig ( since control rig works in its own space ), so what’s needed on your end is the literal translation of those values.
From World is a node that you can use to solve this issue in Control rig, and eventually you can add rotations/translations/offsets directly in control rig quite easily by using local space.
Thats very helpful. The trackers i’m using are the HaritoraX2 and the world transforms seem to be pretty spot on. I have narrowed the issue to one of 2 problems. its either offset of the trackers or the control rigs local spaces. The offset would make sense as the rotational anchor of the joints is the bones tails and heads connection point within the mesh. And the trackers themselves are mounted to the outside of the body. so that difference may be causing some weird rotational problems. I’ll look into the From World node though, that might help remedy the problems. Would you be willing to share screenshots of your set up? I’d like to glance through it and see if i can glean some insight into why mine might not be functioning as expected.
Oh yes, the offset for the tracker needs to be take into account, especially for rotation.
What I ended up doing is to literally measure the distance between ( for example ) the head center and the position of the tracker, and use that as an offset.
By doing so I would get the offset center of rotation to drive the rotation/location of the head.
Below a screenshot of the transform value ( raw from the tracker ) applied to a control in Control Rig.
That is very helpful. I will do some testing with this very soon.
Ok after some extensive testing I have stumbled onto the root cause of my problems and likely a similar problem others might be facing. There is a major bug in 5.6.1 with either the control rig node in the anim graph or the control rig editor “controls”. The controls are not consuming the translations of a motion controllers input through the control rig node in the AnimGraph, only the rotations. I have proven this bug by using custom transform variables and feeding the transforms into those custom variables in place of the control rig control pins rather than the control pins. And then driving the IK directly with the variables. This does introduce new issues, however I think I can use these variables to set the transform of the of the controls similarly to how you’ve done it in the example above. I’ll do more testing today, if this solves the issue I’ll provide screenshots of my workaround and set that as the solution to this forum post.