I was quite sure it was plugin related, but it turns out our issue was related to nested blueprint structs, if you are having the same issue try the method I outlined in this thread.
With this our problem was solved
https://forums.unrealengine.com/t/project-upgraded-from-4-27-to-ue5-instantly-crashing/517658/2