Yeah, I think it might be a bug or something that the “skip assigning gamepad to player 1” used to work in previous versions of UE and now is not working in 5.1.x.
The fix only works for two gamepads. Keyboard and the first gamepad controlled the first player no matter what I did.