I know this is a bit of an old topic but posting anyways -
As of this writing (4.20.1) UE4 doesn’t by default have server names in OnlineSubsystems. For Steam in particular, it uses OwningUserName as the server name (see line 154 of OnlineSessionAsyncServerSteam.cpp). You’ll have to modify the engine to allow for custom Steam server names, whether that be adding in a method to set/retrieve the server name, or adding a custom session setting to populate it.