Idle character is pushed by objects at intervals [With video]

Hi there,

I’m having this problem where I have a door that’s a skeletal mesh and when I open this door while staying idle, the door pushes me but it looks very “jerky” or skippy as if the collision was updating the position every second or so.
This problem doesn’t happen while moving, during that it looks smooth.

I’ve read some other people having this problem but the posts were kinda old and I was wondering if anyone has a workaround to solve it. Most of the workarounds I’ve seen were possible because of them using a timeline to move the door, but I’m using actual animations (I need them), so that doesn’t work for me and none of the possible solutions I have tried work at all.

Here’s a link to a video showing the problem with a static mesh (It’s the same problem as with using an animation with a skeletal mesh)

Thanks in advance for the help.