You can grip things with physics constraints instead of direct parenting… Unlike Unity which has a fixed update physics thread we can’t really do reliable direct velocity control for held objects so that is the best method.
The in engine Physics Handle does just this and multiple people are using it in VR to “physically” hold objects, in the back end it is a physx kinematic actor with a constraint to the held object, then it just sets the kinematic actors location each tick to where you want it to move the object.
I’ll also note that physics based grips are a common feature in many VR games now and that there are multiple ways to avoid disassociation with limbs. Some of the MOST played VR games make use of physically based gripping.