SetActorLocation not replicating

Here is a video for reference: SetActorLocation Not Replicated - YouTube

For some reason when I call SetActorLocation on my pawn it does not get replicated to the clients. As you can see in the video, the normal movement is replicated just fine (which is done through AddMovementInput). But when the player grabs a piece, I do some math so that moving the mouse rotates the pawn around the piece instead of rotating the camera. This movement is done through SetActorLocation.

My pawns (the white pyramids in the video) are derived from ADefaultPawn

class PROCEDURAL_API AMyCharacter: public ADefaultPawn

and within AMyCharacter’s tick I’m simply just doing this when holding onto a piece

void AMyCharacter::Tick(float delta_seconds)
{
  ...
  ...
  if (HasAuthority() == true)
  {
     // Update the player`s location 
     SetActorLocation(newPos);
  }
}

You can clearly see in the video that the client pawn’s location is being applied on the listen server, but it is never replicated to the clients, not even the one doing it. Any ideas?

Also If you notice at the end of the video when I fly the client’s pawn back to the other pawns, it takes ~10 seconds for the other clients to see this. Idk if it is somehow related but it only happens after I grab and let go of a piece.