Snap-turning in VR causes crazy velocity to be applied to attached physics objects

I’m using “Add Controller Yaw Input” to change my player’s orientation, in order to implement snap-turning in VR. This works except if I am holding an object with physics active – the instantaneous turn is interpreted by the physics as a crazy increase in velocity. Is there some way to prevent that? I guess ideally I’d like it to instead apply the velocity that would have been applied by smooth-turning the same amount. But it would be fine if there were just some way of having the turning not affect the physics at all.