I did not really change anything about the collision after I started working on this. But my characters just start bouncing around and flying when they collide. Why is that?
I added a gif of the bouncing and my capsule collision settings.
OK nevermind I had an old component in the character class that still had the collision turned on. 
What component was that?