HybridWithSweep keeps it 1:1 until it hits something and then it simulates. Your problem though is that your IK is tracking the motion controller which has no simulation. Your IK should be tracking the actual object itself. They will never be 1:1 as long as you are tracking the motion controller with it and are using physics grips, the moment you push the held object into something or have the constraint weak enough that it doesn’t keep up the hand will seperate.