Well it depends on how compilcated your system should be.
Let me try to explain the simplest model of a boat. It is just an object with buoyancy. So the water
applies a force to the body depending on how deep the body is immersed in water, basically pushing it upwards.
Find out how deep the body is immersed in water (or find how much of the area of the body is immersed) and calculate the force that pushes the boat upwards.
When you add a motor to this, you basically just need to add a force in it’s forward direction to make it move.
That model is super simple, not taking waves into account.
well paddeling instead of a motor would be pretty much the same. Just add the forward force when the paddles reach water and somebody is rowing. As soon as the paddles leave the water, stop adding the force.
Well, forces on sails are rather complicated, but can easily fake it by just taking the wind vector and mulitplying it with an arbitrary “SailForceMultiplier” and then add this force to the boat.
Not sure how much you want to simulate though, you can go pretty deep into this and simulate in what angle the wind hits the sail, add a force on the sail location to tilt the boat and stuff like that.