Horrible VR Weapon Sway With Physics

Horrible VR weapon sway with physics enabled when moving. I use physics in my game on the gun for recoil and a butt load of other stuff so turning it off really isn’t an option. Does anyone know anything I could do? This weapon sway not only looks bad but also causes VR sickness. If anyone has any ideas I would be VERY grateful. I’ve been pulling my hair out trying to figure this one out. Here’s a GIF so you can see what I’m talking about Screen capture - 4774750e9fc94ec7f0357c0e9523ec28 - Gyazo

Thanks for your time reading this.
-Micky