Matching Crosshair target when toggling between TP & FP

Hi guys, I have been racking my head over this. Everytime the player toggles perspective, the crosshair moves a bit to the side. My understanding is, because I change the player rotation in the end, new CrosshairTarget (line trace) shifts accordingly.

Any idea’s on how I can overcome this?

void OnViewModeChanged()
{
if (GetController())

{
	// Match crosshair target.
	const auto Start = ViewMode == EViewMode::VM_ThirdPerson ? FPCamera->GetComponentLocation() : TPCamera->GetComponentLocation();

	FVector Dummy;
	FRotator Rotation;
	GetActorEyesViewPoint(Dummy, Rotation);  // Rotation = GetBaseAimRotation();

	FHitResult Hit;
	GetWorld()->LineTraceSingleByChannel(Hit, Start, Start + Rotation.Vector() * 10000.f, ECC_Visibility, Params);
	const auto CrosshairTarget = Hit.bBlockingHit ? Hit.ImpactPoint : Hit.TraceEnd;

	auto TargetRotation = UKismetMathLibrary::FindLookAtRotation(ViewMode == EViewMode::VM_ThirdPerson ? TPCamera->GetComponentLocation() : FPCamera->GetComponentLocation(), CrosshairTarget);  // Solve for the rotation of the other camera.

	GetController()->SetControlRotation(TargetRotation);
}

//ChangeCamera
}