For anyone to find this, here is how I finally did that:
this tutorial is a base: Networked Physics - Pawn Tutorial | Community tutorial
(but check the latest comments)
I’ve created 2 separate actors with network components, one is with actual control (apply torque) and the second one is without anything
Then I created a APhysicsConstraintActor, constraint the actors, and pass the reference of it into that actor with control.
In actor with control I’m not just apply torque, I also get handle to constraint and control it there as well.
And that’s it, it works smooth and good with average network connection. Hopefully that will be enough for my case 