Are you using projectile movement component for the bullets?
Looks like initial impulse is not being applied correctly.
Can they collide with gun collision, or your character collision?
Try to put them temporary to overlap all, and see how they behave.