Not sure if there’s a native way of handling this. The only thing that comes to mind at this point is to sweep a collider that is slightly larger than the held item and stop updating control rotation as soon as the sweep hits another physically simulating object. Could work but seems janky at best:
This is utterly untested psedoscript I’d experiment with if i got really desperate; fresh out of good ideas.