I solved this by having it remove (in reverse order from 3 going down to 0 in a for loop) the players that I saved in a savegame object file as “Not Joined”. This seems to work pretty well.
Thing is, I needed 4 players before the game starts in earnest and they enter the level with Pawns, but I need to remove those players that are no longer involved once they get into the arena. I think players persist between levels is the reason.