We have the exact same issue, especially this join / leave message instantly happening.
@andrej / @Flak have you tested staying in the game for a while?
In our game it takes 15 minutes for the session to get glitched and unjoinable with 8 players in the game.
With 4 players, takes around 35 minutes. With 2, around 55 and 1 player 120 minutes.