What could be the reason for a random non-synchronous behaviour of a replicated actor?

The below Image shows the position of the Actor (Yellow) on the Listen Server (Left) and Client (right). The motion of the Pawn is being driven by a TimeLine and for the most part replicates fine. However, almost in every play session and at least once, it happens so that the Actor changes its location. When it again moves upon its turn it recorrects it position thereafter. This whole thing potentially can cause an unpleasant experience for the Player.

What could be the reason?

The LudoPlane you see is an Actor blueprint with all static components. Only the Pawns (Red and Yellow color) which are actually not based on Pawn Class but an Actor class have components which are movable for obvious reasons.

Will appreciate help. I have grappling with this issue for long and have tried changing the NetPriority Float value, but still have the issue.

regards,