How to make a ball which rolls from a point to another?

You do still want to use Torque, think of torque as force around an axle to applying it to Z will spin it around the Z axis, you do want to apply it to X an Y. Torque though, needs huge numbers like 2000000