Well, I found the problem. After some tinkering, I noticed that the skeletal mesh would look fine without any animation applied. But as soon as I added an Animation Blueprint, the artifacts would appear.
Turns out my skeleton had a scale of (0.01, 0.01, 0.01) and that does not work well.
I applied the scale in Blender and redid the animations and now it looks correct.
Thanks for the help, Much Appreciated!