Physics handle not working with bones

It appears there is something very wrong when trying to use a physics handle on a ragdoll bone. No matter what I do it heavily dances around the target location, like all kinds of heavy forces are applied to it in different directions.

I basically want to grab a ragdoll with my mouse and move it around in Y and Z. It works fine with simple physics objects without bones but with skeletal meshes it is just a mess.