Update: After some messing around with lighting in the world in editor, it seems that it is more specifically an issue with LODs.
The model seems to cast shadows sometimes now, but it still has the LOD popping issue and that seems to make it stop casting shadows too.
I have no idea how to remedy this as it is just coming from the animations and not the model.