A common problem when you focus one object and then pan/rotate is the distance kept.
It can be really annoying because you have to focus one object and zoom to change the distance.
One good solution to this problem can be to have an “Orbit Distance” slider, when you stop to focus the distance from the slider is used front of the camera.
It’s a common problem in all 3D applications.
EDIT : One current issue on the actual implementation is the teleport of the camera if the camera is after the focus point and then you orbit.