Construct a camera component on a non pawn?

I’ve been trying to use a camera on a non-pawn actor with no luck.
I currently have the player controller, player camera manager, game mode base, and pawn subclassed. Ideally I would have the following structure:
MyPawn (possessed by MyPlayerController) spawns and holds a member pointer to a CameraActor which holds the CameraComponent. This doesn’t seem to work though. The only thing that works for me is if the Pawn itself holds the CameraComponent. The reason for this is because i may need multiple cameras and it seem’s inappropriate to constantly possess/unpossess from the playercontroller everytime the camera needs to switch. So I want the player controller to keep 1 pawn possessed and have that pawn responsible for setting the current view target

Any advice greatly appreciated.