My Character starts flying when aiming down

It’s hard to say anything without seeing blueprints.

But I assume that the weapon and character collisions are making each other move? Or maybe the cursor? First one is most likely.