I have found the issue, it seems that you can set the sort priority of the particles individually. This fixed it for me.