Making Realistic Player Interaction in VR (Tutorial)

How do I dynamically change COM?

My problem is no matter how I set the center of mass offset off the hand, it still translates (not just rotates around its center) as I apply Angular Stiffness = 30,000 It looks like as if it is doing orbital motion around the controller when the user slightly rotates the controller.