Gamasutra: The Art & Business of Making Gamesspacer
Measuring Responsiveness in Video Games
View All     RSS
October 22, 2014
arrowPress Releases
October 22, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Measuring Responsiveness in Video Games

July 16, 2008 Article Start Page 1 of 3 Next
 

[How do you measure response time in games? Neversoft co-founder Mick West follows up a previous article on responsiveness with a cunning 'how-to' about using a digital camera to track responsiveness - benchmarking games from GTA through Heavenly Sword along the way.]

In this article I suggest that the specifications of a video game should always include a measure called "response time" (also called "lag", "controller lag", or "input latency").

Response time is defined as the time between the player using the controller, and the results appearing on the screen.

Example: Pressing the trigger button on the controller fires a gun on the screen. Video game response time can be measured with a cheap digital camera, and I explain how.

The Problem

The "feel" of a game is in large part described in terms of how "responsive" it is. Very often a game will be described as "laggy" or "sluggish", and by contrast other games will be "tight" or "fast".

I have previously described the technical reasons behind games lacking responsiveness, but I offered no way of measuring the response time, and so the developers have to rely on their own assumptions about the way they read the controller and present the results, and combine that with the subjective assessments of the test department.

Having an accurate way of measuring response time allows the developer to both verify their own assumptions (hence detecting bugs that are adding to the response time), and to provide an objective reference to the claims of the testers regarding the "tightness" of the game.

Perceptions of changes in small variables like response time can vary by individual, and being able to measure it objectively will allow you to see if it has actually changed, and by how much.

Game developers also have to make the decision of whether to go with 60fps or 30fps. 60fps will generally have half the response time of 30fps, which can be a deciding factor (along with the smoother motion, which is visually more appealing on fast moving games). However for some games there are other factors that influence the response time.

Having an accurate way of measuring the response time allows the developer to more accurately and objectively make a decision on if 60fps is necessary, or if they simply need to tighten up their 30fps game.

Measuring Response Time

Measuring response time is very simple, and consists merely of videoing the screen and the controller at the same time with a video camera that records at 60 fps, and then playing this back and counting the frames between button press and the screen response.

The camera I use is a Canon Powershot SD800IS, a relatively cheap camera. You need a camera that supports 60 fps (frames per second) recording. This may be listed as "Sports Mode", or "Fast Frame Rate".

I believe all the current line of Canon Powershot SD cameras have this feature. This is a very popular brand of camera, so it is highly likely that someone on your team will own one.

Set the camera to movie mode, change to to the appropriate mode for 60fps (press the "func" button on the Canon SD and select "Fast Frame Rate").

Then set it up to point at the television. You don't need to get the whole screen in shot, just enough so you can see the game actions that will result from your button presses. Make sure that the controller is roughly in focus and hold it angled so that the button presses can clearly be seen.


Article Start Page 1 of 3 Next

Related Jobs

Avalanche Studios
Avalanche Studios — New York, New York, United States
[10.22.14]

UI Programmer
Avalanche Studios
Avalanche Studios — New York, New York, United States
[10.22.14]

UI Artist/Designer
Wargaming.net
Wargaming.net — Hunt Valley, Maryland, United States
[10.22.14]

Lead UI Engineer
Wargaming.net
Wargaming.net — Chicago, Illinois, United States
[10.22.14]

Lead UI Engineer






Comments


Martin Tremblay
profile image
Very good article!



I would love to see this done with MMORPG interfaces. Everyone claims that WoW's interface is the best, and all other games are sluggish compared. I wonder if WoW is just less laggy! I`m sure the stats on that could be pretty interesting :)

Bob McIntyre
profile image
Since when do game developers "have to choose" between 30 and 60 FPS? Forget about PC games, for the moment...almost no game runs solidly at a fixed framerate even on consoles, unless the console is grossly overpowered for the game and actually has idle time between frames.



Also, Martin, I don't think it will make WoW less laggy because the primary bottleneck for lag in an online game, especially an MMO, is almost always the network connection. Even playing Warcraft or Starcraft, you can notice a slight lag difference between single-player and BNet multiplayer.

Martin Tremblay
profile image
Yes, I agree Richard (Thank you for the comment! :)), that it being online creates its own set of lag. What I was trying to get at is compared to other MMO's, WoW's interface responds better, and seems a lot less sluggish. So if you play WoW and EQ2, both at 100ms Latency, and WoW's interface does respond faster, then it could be that WoW just is more responsive in the way some console games are more responsive. Since both games are online, with the same latency, online shouldn't be a factor.



Maybe I'm missing something though :) I'm not an expert or anything haha.

Jeremy Alessi
profile image
Another solid article that every game programmer should read. Thank you!

Daniel Lam
profile image
That's a pretty good idea! Though I have to point out that using the PS3 XMB may not be the most accurate calibration tool, unless it's official word that the XMB is indeed instantaneous.

In addition, TVs lag differently depending on how much processing is required at different resolutions too. Be sure to set your XMB to the resolution you are testing with (typically 720p)



The best way to measure the response time of your TV is to hook up a PC with DVI/HDMI-outputs in mirror mode, and run a video that has a countdown timer. Use the camera to take snapshots and you'll see the exact number differences between the PC outputting and the TV outputting.

Source : http://hdtvlag.googlepages.com/ourtest

James Williams
profile image
Hi, I have worked on both Goldeneye and Perfect dark franchise plus I'm an avid quake player who is used to playing his FPS games at 120 frames per second (and 120 hz on the monitor) so I was interested in this article as I feel the validity of your claims are missing from a lot of console games and are partly why I not longer work with game consoles.



I think the problem is a lot of devs will always sacrifice frame-rate for better visuals because those gains are a lot more obvious to publishers and consumers, it's also fair to say that some games can get away with being more sluggish in terms of response time.



FPS and Driving games (this includes tony hawk:P) where the life or death of a player is heavily focused on the performance of the game so yeah its vital there to keep it tight, however other games have mechanics where that level of continuity is not required and I guess this is where the games start to feel like a bag of nails.



I would like to see devs take a note from the PC world and put in more video options, so for example if your game uses real-time this and real-time that then at least put in the options so "real-gamers" can sacrifice graphics for a smoother gameplay expereince.



you can even make it cheat based so such options are only enabled afte the game has been finished, so you can still have your great looking game upon release:)



Lastly I would say that however you prsent your findings, anyone worth their salt in games should know if a game feels like crap and if they still release agame knowing this, then maybe its best they carry on and hopefully disappear :)



I find it scary that in this gay and age peple are aiming for 30, it should be 60 FPS and if you're dropping frames, then you should be rethinking your level deisgns and art in that area because unless you have crap game engine then it should not be hard for most people to obtain a crisp expereience.



lastly, I don't know how long this will take to filter into the console market, but in the pcmarket it is possible to increase the polling rate of usb devices.



They run at 125hs as standard, but you can overclock to 250, 750, 1000hz, which make the general movement feel a lot smoother.

Steve Huckle
profile image
How about measuring a game like Guitar Hero with the lag of the Audio? That would ignore frame boundaries, and give you a very accurate measurement of the lag. Plus it's a music game, so it might be important :0



Same method, just record the button press and the audio output of the game, then look at the difference in the attack portions of the sound.

Aaron Claussen
profile image
to James Williams,



While I see your point on frames per second being important in a fast twitch game, such as Quake, because they allow for a faster response time you have to take something else into account. YOU are able to run games at 120 fps but many gamers are not. This could, arguably, give you an edge against other players. While this may help you get kills it also kills the spirit of competativness. By keeping games capped at a certain target fps it helps to keep the playing field level by not allowing any one player to leverage their hardware in such a way as to give themselves a better response time. Granted there are other factors, especially things screen resolution, but from a purely "twitch movement" perspective it helps to keep the playing field level.

John Mawhorter
profile image
This is exactly the kind of hands-on article Gamasutra should run more often. You should do tests of a bunch of game genres comparing responsiveness to critical reception.

Kenneth Baird
profile image
On the pc, sometimes bus transactions can delay USB mouse information (not sure if it affects ps2 mice). I've seen it in particular with sli / crossfire rigs where buffers have to be copied across the link (not properly clearing an offscreen surface for instance), or if excessive texture uploads happen in a frame.



Sometimes your framerate will be fine but there will be a good half second lag in mouse movement to actual movement on the screen due to the above.

jessica winslet
profile image
Slowly but surely the top LCD and Plasma Televisions are getting to grips with combining TV PIctures and gaming technology. Something that should have been straight forward to do for some reason always got held back. Interesting article

http://www.site-de-bingo.fr

Wa Luigi
profile image
Useful information but I completely disagree that reviewers should start using these methods to measure lag and include this information in the review. That's like including the amount of film grain in a film review. While this data is very useful for the people creating, it is not for the player/viewer. For better or worse, lag is sometimes used for gameplay feel (i'm guessing this is the reason the turn in Heavenly Sword is slower than the attack) and it is my opinion that instant response is not the answer for every application. So this sort of "under the hood" data should stay with the developers... please GOD... don't let game reviewers bust out their Lag-O-Meters.

Mick West
profile image
Wa Luigi, while there are a few times when a developer may deliberately introduce a delay into a move, the majority of the time the player (and the developer) wants the response to be as quick as possible. Take for example firing a gun - it's very difficult to see any benefit from adding a 1/5th of a second delay between pulling the trigger and the gun firing (as we have in GTA-IV). The same applies to movements such as simply walking around.



And where a delay is inherent in the move, such as a power attack with a sword, the player still wants SOME response when they press the button, at the very least so they feel the game is responding to them - such as starting an animation.



I think you would be VERY hard pressed to find a game where the developers had DELIBERATELY added a short period of total unresponsiveness after particular button presses. In just about every case the lag is inadvertent, and perhaps unnoticed by the developers. Bringing lag forward as a measurable issue will help ensure developers pay attention to lag, and minimize it where possible.



Even if you accept that in a game such as Heavenly Sword there are particular moves that have additional lag, there is still the issue of the baseline lag - the shortest lag with which the game responds. This lag is often due to underlying technical issues I outlined in another article, and is generally something that is detrimental to the gameplay experience, and something the developers would prefer did not happen.



http://cowboyprogramming.com/2008/05/27/programming-responsivenes
s/


none
 
Comment: