I’ve been playing Paragon for sometime now and I’m quite interested in knowing how Epic did the lobby and starting the game.
The Main Menu of the game, is that a Master Server we connected to?
Is Lobby part of Main Menu or is it completely another map
Assuming Paragon uses AWS, can Epic please shed some information on how they handle dynamic instance creation? For example, in a Coop vs AI match when 5 players are joined and ready for match, how does the Engine handle it? Does AWS create a new instance with the dedicated server and the Engine move all the players to the new server? Actually this is the part I’m most interested in. How does the Engine know the IP of the newly created instance and how it moves all the players to the new server?
I know these are some core development thingies but if possible, I’d really appreciate some detailed answers on this subject especially on how Engine interacts with AWS and how Engine creates new instances and connects with them.
Hi ryanjon2040,
Please do not bump threads that have gone unanswered for at least 4 days. This clutters the forum and makes it difficult to parse through. If you have not heard from anyone in 4 or more days, please feel free to bump the thread once.
I am working on an similar network architecture and I am intrested too. Would be great if EPIC could give us a little insight in the whole networking stuff.
This is a humongous question, but I’ll do my best to describe most of the high level details.
As many of you know, every place in the game is a “map”. The engine can run in several modes (client/dedicated/listen/standalone).
In this case Paragon has several maps (main menu, lobby, Agora, transition map)
transition map is an empty map used to garbage collect between real maps when using seamless travel
connecting to a server always requires a hard travel, once connecting travelling between maps can be done seamlessly
– hard travel requires a loading screen
– seamless travel ticks the engine, keeps network connections open, can even work with Actors/RPCs
leaving a server back to main menu is also hard travel
In Paragon
Servers:
dedicated servers are running, advertising with our session backend (IOnlineSessionInterface), but devoid of game configuration, waiting for a client to contact them via beacons
probably a little much (and possibly I’m not allowed) to go into details about our cloud hosting, but Linux instances are running, monitored, and autoscaled
the important part is that they are preallocated, not spawned on demand. We keep a buffer of servers and spin up/down based on that
Clients:
start in the main menu as a standalone map with no networking running.
the party system uses XMPP to connect social party members together for an evening of gaming.
The party API is a higher level construct that provide UStruct and other simple POD reflection of data to other party members to share
party state (UParty, UPartyGameState)
party member state (UPartyMemberState)
signaling what server to join
signaling matchmaking progress
Matchmaking occurs, using the session interface and the beacon classes to find and join a server
party leader find preexisting games already being filled to join first
failing that, empty servers are contacted by the party leader, told to configure themselves for the right game type, and load into a lobby map ready to receive new players
beacons (UOnlineBeacon) are used to make reservations on the server
– in a high concurrency game, prevents dozens or hundreds of players from rushing the same server
– provides a guarantee that one particular server will make/have room for you
beacons are used to keep all clients in contact with the server until the server is full of players
– makes searching for another match easy if things go wrong
– real server travel is expensive (hard travel and therefore loading screen, full map load, etc) and hard to recover from gracefully (ie automatically getting back into matchmaking)
server fills up on the beacon, notifies all party leaders to tell their party members to connect “for real”
Regular Unreal networking occurs
connect to the lobby map as any UE4 game would (ClientTravel(URL))
normal server logic, replication, RPCs, client logic
additional xmpp chat lobbies for teams
draft lobby ends
seamless travel into our game map
game ends
players return to main menu, keeping their xmpp connections intact the whole time for both rooms
post game ends, team chat is destroyed
Repeat
Super high level, hope it helps, I can try to field more questions, we’ll see how it goes.
This may be a little dated, but here is a linkto a GDC talk I helped write (but didn’t give) that describes how Gears of War 3 (and Judgment) did this. It was my first iteration on dedicated servers. The hosting information isn’t as relevant, but some of the work around servers certainly is.
[MENTION=394][EPIC] Josh.Markiewicz[/MENTION] Thank you very much Josh! I really appreciate the response
I am using GameSparks as our backend platform. The current status of my project is I am able to login using user credentials and ping (connecting to data centers message…) my AWS instance (using TCP only. For some reason UDP is returning timeout error). After that player goes to main menu, choose their character and initiate matchmaking service (using GameSparks). I am able to spawn a new instance upon receiving the first player and save that information to the first player in the lobby but this is not a good procedure because the instance itself takes few minutes to spin up. But still, I am able to get the IP of the new instance and connect to that after it is up and running.
So I am very interested in learning this procedure you said here:
Can you please give a little more detailed explanation on how to do this?
I think that he means that for example: If you got 100 players online (maybe ~10 idling), you start up some server instances, for example 7 instances (= 70 persons) in order to avoid on-demand start-up time. Imagine having to wait up for a VM or container to initialize each time a match needs to begin. Some try to optimize start times much and other’s just go with buffers. Then basically keep spawning server instances (and keep track of em) in advance…how much you need to spawn at which time is determined by observing your player base activity, national holidays etc
That’s basically it. We monitor the number of online players and keep a 20% buffer say. This automated process knows the total number of player and current server availability. Server processes start up on demand based on this auto scaler. When we start to see population rise, we add more servers, when population declines, we ask servers to shut down when able (typically at match end). It can take into account the velocity/acceleration of the population growth and spawn servers faster.
Our backend matchmaking service knows these numbers, you’d need to track them somehow via GameSparks or via your own method from the cloud.
Thank you @Zarkopafilis and [MENTION=394][EPIC] Josh.Markiewicz[/MENTION] :). Really helpful information. I was reading and researching about Game Sessions but I think OnlineSubsystemNull wont work through internet…right? Even if I have a dedicated server advertising a session (via Host Session) on AWS instance, the clients wont be able to find (via Find Sessions) it right? I saw very little information about OnlineSubsystemMcp but can’t find any information regarding it.
That is correct. The NULL interface is a mocking interface. It only handles basic LAN play and requires client side filtering of search results. Many of the interfaces don’t do anything other than return a completion delegate to make gameplay code less messy to write when you don’t have a real online subsystem.
OnlineSubsystemMcp isn’t released to users yet. It is an internal service we use and isn’t ready for mass distribution.
Your choices at the moment are Live, PSN, and Steam. Or you could implement your own. I’ve heard people have success with RakNet, but we don’t support it directly.
