The lag-fighting techniques behind GGPO’s netcode
Programmer Tony Cannon explains how his GGPO netcode can make timing-based skill games more responsive in online play, in the September 2012 issue of Game Developer magazine.
Every single millisecond matters in an online multiplayer action game. Take Street Fighter, for example; any fighting game enthusiast knows that one opponent is sitting behind the other screen, and the other is the lag that makes playing highly timing-sensitive games no fun. In the September 2012 issue of Game Developer magazine, programmer and fighting game community leader Tony Cannon takes a deep dive into how the techniques behind his predictive netcode, called GGPO (short for "Good Game, Peace Out"), can make both classic and modern games look and feel more responsive -- even if you're playing with someone in another continent.
Identifying the chokepointThe traditional way to add online support to a multiplayer game can lead to "squishy" and "laggy" experiences, says Cannon, and for certain game genres that can ruin the multiplayer experience entirely -- especially if you're trying to add online multiplayer to a port of an older game that didn't have it before. Cannon explains:
The most common method of bringing multiplayer games online is to run a separate simulation on each console and keep them synchronized by ensuring that each simulation gets exactly the same inputs. If the simulations are determined solely by the inputs, playing the same inputs on both consoles will result in the same output. This method has a lot of plus sides to it; computers are by nature deterministic and most arcade games use integer math exclusively, which is very well behaved from processor to processor, and the developer is also completely isolated from the details of the network implementation. Aside from making sure the game runs deterministically, the engineers and designers implementing the gameplay can be isolated from the details of the network engine. This means you can retroactively add online multiplayer support to games that have already been released. (If you can provide a machine emulator for the original hardware the game was written on, you don't even need to recompile it!) Finally, this method has no server-based components aside from a method required for basic matchmaking, which is usually provided by the console manufacturer.
Unfortunately, this method also comes with one major drawback: These games are usually designed to sample the controller input before every simulation state update. A simulation frame in the game cannot execute until all the inputs from remote players have been received. In practice, this manifests itself as an input delay equal to the time it takes to send a packet from one console to the next. In short, the games lag. The instant response the game was designed for is replaced by a sluggish and squishy control that can ruin the players' experience.
In many cases, the delay introduced by the networking layer can completely change the feel of the game. In Street Fighter, we call them "lag tactics": using the knowledge that by the time your opponent sees your move it will be too late to respond. To many, including myself, lag tactics greatly influence the competitive experience of the game online.
GGPO's solutionIn order to minimize lag, Cannon designed GGPO to keep the local player's inputs responsive, predict the remote player's actions, and subtly roll the game back when it receives the remote player's inputs and finds where those inputs were not predicted, leading to more responsive gameplay. In the article, Cannon explains:
I wrote GGPO to try to do a better job of solving the lag problem, without sacrificing all the other great properties of this method. Instead of adding an input delay to the entire simulation, GGPO gives the local player immediate control over his avatar by hiding the latency of the connection in the start-up of each remote player's action. This does a much better job of faithfully reproducing the offline experience in the online game.
GGPO uses speculative execution to eliminate the perceived input delay for each local player. Instead of waiting for all inputs to reach a player's simulation before executing a frame, GGPO will guess what remote players will do based on their previous actions. This eliminates the lag experienced by the local player created by the traditional frame-delay method. The local player's avatar is just as responsive as when playing offline. Though the actions of the other players cannot be known until their inputs arrive, the prediction mechanism used by GGPO ensures that the game simulation is correct most of the time.
When GGPO receives the remote inputs from the network, it compares the predicted inputs to the actual ones. If it finds a discrepancy, it rewinds the simulation back to the first incorrect frame, repredicts the inputs for each player based on the updated input stream, and advances the simulation to the current frame using the new prediction.
Grab the September issue of Game Developer magazine to get the full scoop on GGPO. Also, check out this issue for tips on how to tackle development weaknesses and make your studio a more successful one, as well as a postmortem of experimental PlayStation 3 music game PixelJunk 4am. Game Developer is available for purchase or subscription at the GDMag web store, or you can download the Game Developer iOS app.