Measuring Responsiveness in Video Games
July 16, 2008 Page 3 of 3
I tested various other games, with various results. I'll list the results in 60ths of a second, and the results have been adjusted -2 to account for the plasma lag. All games are on the PS3 unless noted otherwise, and I've included the PS3 system menus and GTAIV for reference.
Games that run at 60 fps:
- PS3 System menus: 3/60ths
- Guitar Hero III (Xbox 360): 3/60th
- Ridge Racer 7: 4/60ths
- Virtua Tennis 3: 4/60ths
- Ninja Gaiden Sigma: 4/60ths
- PixelJunk Racers: 4/60ths
Games that run at 30 fps:
- Genji: Days of the Blade: 6/60ths
- Tony Hawk's Proving Ground: 8/60ths
- BlackSite: Area 51: 8/60ths
- Halo 3 (Xbox 360) : 8-10/60ths
- Skate: 10/60ths
- GTAIV: 10/60ths
- Harry Potter and the Order of the Phoenix: 10-14/60ths
- Heavenly Sword: 7-18/60ths
The first question that arises is: why are there no PS3 games with a response time of 3/60ths? The PS3 UI runs at 3/60ths response time, so it's quite possible. Why do all the 60fps games run at 4/60ths?
Still, that's still pretty good, and those games all feel quite responsive. Guitar Hero III on the Xbox 360 manages a very impressive 3/60ths - very important for that type of game, especially when it's actually 5/60ths on many TVs.
Next, why so much variance between the 30fps games - ranging from 6/60ths to 18/60ths?
It's worth noting that while EA's Skate seems like it would be a bit sluggish at 10/60ths, it actually feels quite responsive, in large part due to its use of stick gestures for input, as the movement starts before the gesture is complete, yet the player mentally synchronizes it with the end of the gesture, and so it actually seems very responsive.
Then what about the games where the lag varies? Harry Potter is 10-14 which is bad enough, but Heavenly Sword is an astonishing 7-18. It takes 7/60ths to start an attack, but 18/60ths to start to turn around (See movie: mvi_4263w). Clearly something is wrong there. I would consider that a bug.
It's sad that the programmer made the effort that allowed for a 7/60th response time, but then someone else messed up down the line, making the turn take nearly a third of a second. Halo 3 is another example, with the shooting and moving being 8, but the jumping being 10.
However, the fact that Genji runs at 30 is barely noticeable, and does not detract from the game at all. In part this is because of the way the camera moves smoothly through the scene with very few rapid pans. It's also because of the low contrast graphics and motion blur. But it's also because Genji's response rate is 6/60ths, very similar to Ninja Gaiden's 4/60ths.
Ninja Gaiden is a faster paced game than Genji, with the main character jumping around rapidly, and the camera tightly following him - so the graphical benefits of 60fps are more apparent. The low response time feels good as well.
However, since Genji also has a low response time, it would benefit very little from running at 60 fps. As it runs at 30fps, this gives the designers the opportunity to put more graphics, enemies and special effects on screen, and reduces the pressure on programmers and artists to constantly strive to maintain 60, which can be a difficult factor in development.
Conclusions and Suggestions
Games that run at 60fps all seem to have a response time to 4/60ths, and while 3/60ths is possible, 4/60ths is a very good response time.
Some games running at 30fps have a response time of 8/60ths or 10/60ths (and some peak even higher). Genji shows us that a response time of 6/60ths is possible while running at 30 fps. 10/60ths can be too long, especially when combined with the processing delays in flat panel TVs which can push it up to 12/60ths. 1/5th of a second (200ms) is too long to wait for a gun to fire, and introduces annoying sluggishness when moving around or steering a car.
Some games have an inconsistent response time. Heavenly Sword varies from 7 to 18. If the system is capable of 7, then all moves should start in 7. Developers should verify ALL their response times, as other factors, such as animation, might be creating lag in specific places.
I suggest that game developers use this simple technique to measure the response time in their games, at least to verify that their assumptions are correct. If they are running at 60fps, then they should not be above 4/60ths. If they are running at 30fps, then they strive to duplicate Genji's responsiveness of 6/60ths, and certainly not slip below 8/60ths. I suggest that they keep in mind that flat panel TVs (which are probably a majority in gamers households, and certainly in game reviewers' households) add an additional 2 frames of lag, which makes it EVEN MORE important to keep programmed lag to a minimum.
I also suggest that game reviewers begin to use this technique to measure lag, and to include the measurement of lag in their reviews. While the subjective views of the reviewer are important and valuable, an objective measurement of response time would be a very useful additional piece of information for the person considering buying the game.
Placing this information in a game review would encourage developers to produce more responsive games, which benefits everyone.
Page 3 of 3