VR joystick: Grab/Hold stick, not push

Hey gamedevs. I would like to grab onto the joystick and move. Currently I have physical constraints and force moving it but that’s not practical. I would like it to feel real so the user is holding and moving the stick in the directions.

Any ideas? Using the VR template.

296938-annotation-2020-02-26-165712.png

Video here:

Woow what is that thing …want to put hands on it link