I’m going to answer my own question here by saying that by default, XInput has a hard limit of 4 physical gamepads.
You can have keyboards alongside 4 gamepads if you can implement it, but that’s not good for local party games.
The only way to have more than 4 players, is to implement DirectInput, or to download a plugin that does this for you.
Don’t waste any time trying to get more than 4 controllers working with anything other than DirectInput, it will not work.
There are reports from other games out there that you can “daisy chain” controllers (not sure what they mean) or use 2 wireless receivers with 4 controllers each, but this is an unfeasible and unrealistic expectation of gamers who are wanting to party.
This is a link to helpful DirectInput information:
https://msdn.microsoft.com/en-us/library/windows/desktop/ee418273(v=vs.85).aspx
And a quick google search will show various plugins that enable directInput.
It is also worth noting, that apparently directInput and XInput can be implemented side-by-side so that you can still have XInput controller functionality on controllers that use XInput, while also being able to use DirectInput controllers at the same time for extra players, just with limited functionality on those controllers.