Guerrilla Multiplayer Development
September 13, 2012 Page 2 of 3
3. Use and abuse Java even more. To rapidly implement the Quake III network model, I made extensive use of Java's runtime annotations and reflection. Annotations were used in game-entity classes to mark fields that could be passed over the network. The client and server assemble UDP packets based on a game-entity's list of annotated fields. Because code is shared between client and server, I only needed to make network protocol changes in one place.
4. Matchmaker service. An extra service is needed to group players into games, communicate with the game servers, and keep track of player stats like XP, cash, and inventory. Amazon Web Services provided everything I needed to develop and deploy the Matchmaker Service including:
- EC2 load balancer
- Elasticache and Relational Database Service (RDS) for the persistence layer (essentially Memcached and MySQL managed by Amazon). From memcached.org: Memcached is a "free & open source, high-performance, distributed memory object caching system."
- Simple Storage Service (S3) to host game assets, like item images seen in the ErnCon Store, and the server WAR and JAR files for the matchmaker.
5. Easy to deploy servers. Although Amazon provides many tools to help develop and deploy the ErnCon backend, I still had to do a lot of work to ensure that provisioning new servers is as painless as possible.
Currently I have a custom Amazon Machine Image (AMI) that on startup fetches the latest matchmaker service WAR from S3. Game servers do the same thing, except they fetch the latest game server WAR.
Game servers automatically register themselves with the matchmaker, so the matchmaker can start creating games and sending players to the game servers. When updating server code, my deployment procedure is simply:
- Upload updated matchmaker and game server WAR and JAR files to S3.
- Run the EC2 Instance Launch Wizard to create a new EC2 instance using my custom AMI.
- Wait for the instance to start and verify that it is accepting players and games.
- Repeat for all geographic regions.
Overall, provisioning a new game server takes less than five minutes.
The multiplayer architecture of ErnCon is something I'm proud of especially since most of it was developed in my spare time. Next step: network protocol.
When designing ErnCon's multiplayer architecture, a major requirement was that the game be playable over 3G -- Wi-Fi should not be required. Playing a game should be as frictionless as possible -- ErnCon cannot ask a player to exit the game just to ensure he is connected to a wi-fi hotspot. Also, forcing a player to be tethered to a home wi-fi connection is the antithesis of mobile gaming! To get ErnCon playable on 3G, I had to consider the following throughout development:
1. Playable with typical 3G data rates. Although a concrete minimum and average data rate is not specified for 3G, research suggests a minimum 384 Kbit/s the data rate.
2. Playable with typical 3G latency. Hard data on typical latency is difficult to find. Empirical testing on T-Mobile and Verizon using a ping app showed 300ms round trip with occasional spikes to 500ms or more seemed to agree with information I could find digging through forums and articles. 300ms isn't good but with clever client-side prediction, could still be playable.
The Quake III Networking Model
Please familiarize yourself with the Quake III Networking Model before continuing with this article. I assume you are already familiar with concepts such as UDP networking. Some of the key points of the Quake III Networking Model are:
1. Server tracks what data was sent to whom. The server does a lot of work tracking what gamestate has been sent to every client. This allows the server to save bandwidth by delta compressing data.
2. Delta Compression of data. This is a fancy way of saying the server will only send a client data that has changed from the last acknowledged state. For example, if I keep track of a spaceship's x,y coordinates and facing, if the player is just moving up (along the y axis) the server only needs to send packets containing updates for that player's y coordinate.
3. Implicit handling of lag. Lag spikes or other UDP data-loss is handled by the server continuing to pump out network updates based on the last acknowledged state received by a client. This removes the need for a separate reliable acknowledgement channel.
Page 2 of 3