Physics Objects get pushed by Player Character using default Movement Component

Thanks for the reply, I tried your solution but the pendulum still gets pushed by the player in some circumstances. In my video I show a before and after implementing your solution to add more context. Also the pendulum is moving through a physics constraint.