How to grab objects in VR while simulating physics and collision for said object?

I have the same problem…