Exclude objects from FOV changes

Hello everyone, I started making an aim down sight system by bringing the gun in the center of the screen and simulating a zoom effect by lowering the FOV. However, I don’t know how to prevent the gun from being affected by FOV modifications, because I’d like to implement multiple scopes with different zooming levels, but to do that the zoom on the gun itself should not be changed.
I tried using Panini projection, but the gun still gets affected (less than before, but still not excluded).
Is there a way of fixing this?

FOV affect the camera that is used.

You need to add a different camera that only is active when you aim