Follow Camera with Lag and Option to move around freely

Hey Guys,

I try to develop a third person pawn with a camera that follows the pawn. The camera has a lag and a max lag distance from 400.
If the player wants to he should be able to rotate the camera around the pawn. Let’s say the player wants to see the character from the side instead of behind, so the spring arm needs to be rotated by 90° on the yaw angle.
If the player continues to move the camera still follows the pawn but always with the 90° yaw additional rotation.
The Problem is because of the camera lag , the camera does now not rotate around the pawn, it rotates around the pawn location and plus the lag distance offset. So the camera does not look at the pawn anymore. See the attached picture.
Is there any way to still have the camera lag and and option to rotate around the player?