Fixing Scope Shift Issues When Switching Between Character and Rifle Cameras

I have 2 cameras, one on the character and the other on the rifle near the scope. To switch between cameras, I use SetViewTargetWithBlend, but after transitioning from one camera to another, the scope shifts. How can I fix this?

ezgif-7-af0902fe60