How to get a consistent offset on a zooming camera?

My screen is split between a “playable” area and a “text” area. I want to be able to click a location in the playable area and have the spring arm/camera be able to pivot around and zoom in on it. Adding an offset to the camera semi-works, but when zooming the object is moved off centre. How can I set up the camera to stay centred on the selected location (relative to the playable area) even when zooming? I’ve attached images to show the problem.