That’s because you’re setting the rotation of the Physics Handle that is, if I understand it right, a mere point in space, and its rotation doesn’t really matter. Instead, you need to set the rotation of the picked object. Whenever you pick it, save it to a variable, say, “Current Held Object”, and set its rotation.