Try this.
Before starting the timeline, get the world location of the object and store it as variable (Make sure this is called only once, don’t use anything like Tick Evet). After that in your lerp, plug in that variable to “A” and have the destination location to “B”. It should work.