Door physics question

I have a simple door set up in a blueprint using a static mesh and a physics constraint. The issue I’m having is that the door jerks violently went the player bumps into it and attempts to push the door the wrong way. I’ve played around with using soft constraints and played with the damping. But nothing I’ve tried so far has curbed this behavior. Is there a way to stop this?

Dont use physics.