I figured it out, so I’m leaving it here for the next person looking.
-Click on the button in the top left of the editor screen. I don’t know what it’s called, but it’s just to the left of the ‘perspective’ button.
-Click ‘advanced settings’ at the bottom of the menu that comes up.
-Go to ‘Level Editor’ / ‘Viewports’ / ‘Look and Feel’
Toward the bottom of that section there are two options of interest:
‘Preview Selected Cameras’ and ‘Camera Preview Size’
The first turns the preview on and off, the second allows you to change it’s size.