Spectator Pawn's looking direction getting reset when unpossessed and repossessed

I am using a spectator pawn as a debug cam to move around my scene. I use a hotkey combination to switch from my player character to my spectator pawn debug cam and back. I am able to move around the scene using the spectator pawn just fine and I am using default values for it.

The problem is when I unpossess the spectator pawn and repossess it again, it reverts back to looking in the direction it was looking at game start. No matter, which direction the spectator pawn is looking when it gets unpossessed, it goes back to looking the default direction.

I was thinking of saving the last direction it was looking in and set it back when repossessed but it seems like the actor’s rotation never changes (its location does though). I really don’t want to manually redefine the controls for the spectator as the default ones are working just fine.

What am I doing wrong here?

I think this is fixed by enabling Use Controller Pitch/Yaw. But I still don’t understand why? Enabling the option causes the spectator pawn BP to show their rotation values in my scene but disabling keeps those values at default. But how does the spectator pawn able to see around the scene either way?