Capsule Collider won't stick to bone

Hello!

I am having a very weird problem.

I have a system setup to simulate realistic collision with the default VR Template hands using physics constraints.

Now, the problem is is with the colliders that are attached to the finger bones.

When moving your hand around and colliding with objects, they work and stick to the bones perfectly! However as soon as you bend your fingers, it just floats.

I attached a few relevant screenshots as well as a YouTube video showing what I mean.

Don’t mind the colliders rolling around I was just testing with one collider for now haha

https://youtu.be/irTKEX3nn2Y