How to make my avatar's hand do not go through the fabric?

[Style3D_CG_official (Style3D CG)]

Have you tried Collison Filter? Or you can follow the steps shown in the picture.