I my problem is solved. In my characterBP under Character Movement look for “Use Flat Base for Floor Checks” in the details panel, check the box and no more stopping at an angle on a ledge.
Still unsure on how to make my character stop a bit before the edge so they aren’t floating over it, but floating over at an angle isn’t an issue anymore. Thinning the radius of the capsule sort of works, but there must be a better fix.
Adij mentioned this in his comment. I can’t seem to mark his comment as answer.