You saying something confusing, you should only have one session for server which clients can join in to and that session should be created by dedicated server
I not sure baout advance sessions but Session system is complitly separate server-client model of the engine, it is OnlineSubsystem feature. There goal is to register the server in to online service you using, but you don’t need online service to run a server as UE4 use normal IP connection (same as Minecraft for example) and online services are only to automatize this process (thats why you never put any IP addresses in game that use them).
If you want to run dedicates server your game need to run in that mode first and im not mistaken sessions should be created by server if it is a server. You can read about it here:
Generally your game should at minimum handle non-session connections, if sessions confuse you, you could just focus on normal connection explained in link above and then put sessions on top of it.