Yep, this would be approximating a spring constraint, and give you the desired effect.
@OP, check out Verlet Integration, you can build a simple spring like @jwatte describes and it doesn’t require complex math.