Is anyone interested in making MMO with distributed dedicated servers?

Community Community & Industry Discussion
, , ,
1

Hi there. I would like to share my open source plugin that enables distributed simulation of dedicated server:

https://www.channeld.org/

Basically, you can combine multiple dedicated servers into one large world to support thousands of concurrent players online. Seamless travel and some basic interactions (rigidbody physics) across the servers are supported too.

The plugin has replaced the underlying networking and replication systems with custom code. In short, it uses TCP, Protobuf, and a code generation solution for the property replication and RPC. The developer should be able to develop in the native way, e.g. UPROPERTY(Replicated) and UFUNCTION(Server) , although some features are not fully compatible yet.

The whole system runs with a standalone gateway service which is also open source. The gateway service interconnects the UE clients and servers, and do jobs like message forwarding/broadcasting and interest management. The design is inspired by SpatialOS. It’s written in Go, which was preferred for fast iteration. I’m also considering rewritten it in C++ or Rust, as there are pain points like slow GC and reflection-based Protobuf marshal/merge.

Performance-wise, you can have 150+ concurrent players running in a single dedicated server stably at 30 FPS, thanks to the code-gen-based replication system and the offloading of interest management to the gateway service. Combing the servers is a different story though - the cross-server handovers add load on the system heavily. As of today, I have tested 600+ concurrently players running in 9 dedicated servers. With some optimization, I believe the CCU number can go up to 1K.

If anyone is interested in this technology, you can have a try - it’s totally free. Feedback is always welcome! If you are also interested in contribution, you are welcome to join the open source community!

I’m also looking forwarding to suggestions on the product. Like, do game devs really need this technology in the future?

https://www.channeld.org/

2 Likes
2

Hi there, your explanation was a little vague(are you breaking the server map up between servers like eve online or have a server farm updating each other like WoW) and are you optimizing for multi threads? but I think you and I are both barking down the same rabbit hole.

I’ve been developing a multi-player video game set up for 3000 players per game instance. I have 2 game server packed builds one for the game instance running in a subnet cluster of 3 5995WX threadripper workstations. each workstation is clustered piont to point(switchless) with 2 100gbs 2 port nics. 1 for out going data to the other 2 nodes and 1 for incoming date. A third nic in each node is used to to connect all 3 nodes in the cluster to the main server running the main game server package(intro lobby for connecting websockets, and load balancing to the cluster, so on)

I’m currently working with rancher kubernetes, kafka, chep, and mongodb. Im still looking at things like am I better off with agones then rancher or what not. My unreal server packages I’m still optimizing for multi threading the three threadrippers. This kind of network architecture makes it easier to scale the hardware of any kind aslong as the server map instance is on the cluster and set up the same way (any identical hardware systems can work) to the games needs.

With (p2p) clustered system latency is low
And multi threading overhead between the cluster is not to bad because only data internal to the game instance is in that subnet. Then all trafic from the main servers web sockets in my case are going through 3 duel port 100gbs nics to the server.

My system takes away the need for large data centers that game server farms need, but still self healing with ceph. And where eve uses a trunk system and splits the map up like UE world partition that can cause latency and lag if to many people are in the same area. Where my fully meshed cluster runs the same as AI LLM systems and you dont need to split up the map and still have load balancing and plenty of bandwidth do to the data being split.