A couple of things initially that I think it could be:
Potentially your collision box is too big at the bottom and pushing it up?
Do you need to include 2 physics constraints? Maybe they are competing with each other?
It has been a while since I have used them!