Ok I found the culprit.
Either in 4.20 or 4.21 Epic changed the PreLogin function in GameModeBase.cpp.
Now it verifies that both the server and the client are using the same OnlineSubsystem type.
I have tried overriding that function in my GameMode to allow any connection. It connects to the server successfully! But now it crashes 
I’ll have to report back later if I can get it to work.
Btw, the super suspicious line is:
UE4 : [2018.12.07-22.53.35:718][539]LogModuleManager: Warning: ModuleManager: Module ‘OnlineSubsystemUNSET’ not found - its StaticallyLinkedModuleInitializers function is null.
I don’t know why it would ever try to load OnlineSubsystemUNSET instead of OnlineSubsystemNULL…