Smashing destructible mesh with fists, not projectile.

I’m trying to have my character smash a box with his fists using a sphere collision on his hands instead of using a projectile but I can’t seem to be able to get a nice result like you can when using projectiles. The box kind of explodes a little on the opposite side. I want to have a cumulative damage result, so he can keep punching the box until it’s destroyed.
Also is there a way to have the player capsule not demolish the box and ignore it?
Thanks in advance