SetOwner() and Actor Replication? (Want to animate client-side only!)

I’m trying to create a “client-side” animation on a StaticMesh actor.
I have a door, which I set myself as the owner, and call a server RPC. This server RPC then rotates the door. (via: SetActorRotation() )

However, I block replication via IsNetRelevantFor(), and check if the RealViewer/ViewTarget is the owner, and return false. I want to rotate it client-side for the owner, as it produces a much nicer animation. Especially if the network traffic is high.

For everyone else, the object rotates as it should. But when using SetOwner() it seems replication happens IMMEDIATELY for the owner. Even when IsNetRelevantFor() returns false for all conditions…

If I had a timer/delay before I do any rotation (server-side) and call SetActorRotation() in a client RPC. Everything works. But why? How do I get the engine to not replicate right away when setting the owner or is there a better way to do this all together?

I would like to avoid baking animations into a mesh…

#include "MyActor.h"

void AMyActor::RepeatingFunction()
{
	svOpenedFully = true;
	bclDoRotate = true;
	Super::Modify();
}

// Sets default values
AMyActor::AMyActor()
{
 	// Set this actor to call Tick() every frame.  You can turn this off to improve performance if you don't need it.
	PrimaryActorTick.bCanEverTick = true;

	/*
	Collider = CreateDefaultSubobject<UBoxComponent>(TEXT("Collider")); 	
	Collider->SetupAttachment(RootComponent, NAME_None); 	
	Collider->SetGenerateOverlapEvents(true); 	     
	Collider->SetBoxExtent(FVector(100.f, 100.f, 100.f), false); 	
	Collider->SetCollisionProfileName(TEXT("Trigger"), false);
	*/

	this->bReplicates = true;
	this->bAlwaysRelevant = true;
	this->SetReplicateMovement(true);
	this->svOpenedFully = false;
	this->bclDoRotate = false;

	ObjectMesh = CreateDefaultSubobject<UStaticMeshComponent>(TEXT("MySuperMesh"));
}



bool AMyActor::IsNetRelevantFor(const AActor* RealViewer, const AActor* ViewTarget, const FVector& SrcLocation) const
{
	bool b = Super::IsNetRelevantFor(RealViewer, ViewTarget, SrcLocation);
	if (!b)
		return false;

	// If we are the owner (the one who opened/used the door)
	// don't replicate data yet, wait for the client-side animation to finish. Then resync
	if (ViewTarget == this->GetOwner() || RealViewer == this->GetOwner() ) {
		return false;
	}

	return svOpenedFully;
}


// Called when the game starts or when spawned
void AMyActor::BeginPlay()
{
	Super::BeginPlay();
}

// Called every frame
void AMyActor::Tick(float DeltaTime)
{
	Super::Tick(DeltaTime);

	if (bclDoRotate == true )
	{
		float DeltaRotation = DeltaTime * 80.0f;
		FRotator NewRotation = GetActorRotation();
		
		if (NewRotation.Yaw < 90)
			NewRotation.Yaw += DeltaRotation;

		SetActorRotation(NewRotation);
	}
}


void AMyActor::Server_ReliableFunctionCallThatRunsOnServer_Implementation(AActor* owner2)
{
	// Bug will happen here.
	// bclDoRotate = true;

	// Delay 3s and everything is fine??
	if( !svOpenedFully )
		GetWorldTimerManager().SetTimer(MemberTimerHandle, this, &AMyActor::RepeatingFunction, 3.0f );

	this->SetOwner(owner2);
	this->Client_UnreliableFunctionCallThatRunsOnAllClients();
}

void AMyActor::Client_UnreliableFunctionCallThatRunsOnAllClients_Implementation()
{
	bclDoRotate = true;
}