I’ve been having the same problem. My character pushes around physics objects too easily. However, one thing sort of worked for me, and might work better for you - try editing the Push values in the Character Movement component of the Player Character. As you can see in this screenshot, just go into character movement and search ‘Push’ under the details panel, and then edit the values to what makes sense.
I hope this helps!
