U5 VR Hand collision question

Hi everyone, I’m developing a VR game and I’m stuck with the hand collision. I have a hand with its animations ( grabbing, pointing etc), in order to not make it collide with its surroundings ( walls, table etc ) I need to make a physics constraint for it with another mesh ( I decided to use an invisible cube which is not rendered when playing), which is ok, the grabbing mechanics are working, but when I associate its animation blueprints the hand lose all the physics properties and goes through the walls again.
My question is, is it possibile to have a physical hand with its animations AND not let it goes through walls?