TPS to FPS while keeping the target in aim

I am working on a system that can switch to aim down sight (FPS) but keep the target in place. Right now I dealt with it by not changing the camera’s location, only FOV and attaching the gun to camera with a smooth transition, but this is not excatly what I want because gun starts circling around the character when I move right or left, and the position is not perfectly fine. So, what I want is currenlt works in every tps&fps games, you play in TPS camera, when you aim, you zoom in and see the scope of the gun, but your aim doesn’t shift to somewhere else, it looks at the perfect position that you were looking in the TPS camera. I tried to maintain this with adding another camera to gun blueprint, and blending to it while setting the rotation of the gun with a line trace from my camera. But it doesn’t work properly since I detach the gun from the character, it starts moving in a weird angle and sometimes roll gets messed up even though I set it to 0. Can anyone give any idea about this?

I have TPP and FPP cameras on my player. I also spawn and attach a camera on the gun in construction graph when the gun is spawned. I use set view target with blend for all camera transitions.

What I found was you need first ensure that the gun camera is perfectly aligned with the weapon. Then you have to tweak the weapons alignment in hand so that it itself is pointing at center screen. It takes a bit of manual adjustment to get this right.