When a PlayerController possesses a pawn, it sets its control rotation to match the Pawn’s current rotation (see APlayerController::Possess() if you have source). I would try to see if the pawn’s rotation is correct on the client when it’s spawned.
You might try using the console commands “displayall Pawn Rotation” and “displayall Controller Rotation” to see what the rotations look like in-game (easier than logging it sometimes).
Controller::SetInitialLocationAndRotation is another thing to look at. That sets the location/rotation of the controller used to spawn the spectator (DefaultPawn). Perhaps this isn’t happening correctly between the server and client.