|
[How do you measure response time in games? Neversoft co-founder Mick West follows up a previous article on responsiveness with a cunning 'how-to' about using a digital camera to track responsiveness - benchmarking games from GTA through Heavenly Sword along the way.]
In this article I suggest that the specifications of a video game
should always include a measure called "response time" (also called
"lag", "controller lag", or "input latency").
Response time is defined
as the time between the player using the controller, and the results
appearing on the screen.
Example: Pressing the trigger button on the
controller fires a gun on the screen. Video game response time can be measured with a cheap digital camera, and I explain how.
The Problem
The "feel" of a game is in large part described in terms of how
"responsive" it is. Very often a game will be described as "laggy" or
"sluggish", and by contrast other games will be "tight" or "fast".
I
have previously described the technical reasons behind games lacking responsiveness,
but I offered no way of measuring the response time, and so the
developers have to rely on their own assumptions about the way they
read the controller and present the results, and combine that with the
subjective assessments of the test department.
Having an accurate way of measuring response time allows the
developer to both verify their own assumptions (hence detecting bugs
that are adding to the response time), and to provide an objective
reference to the claims of the testers regarding the "tightness" of the
game.
Perceptions of changes in small variables like response time can
vary by individual, and being able to measure it objectively will allow
you to see if it has actually changed, and by how much.
Game developers also have to make the decision of whether to go with
60fps or 30fps. 60fps will generally have half the response time of
30fps, which can be a deciding factor (along with the smoother motion,
which is visually more appealing on fast moving games). However for
some games there are other factors that influence the response time.
Having an accurate way of measuring the response time allows the
developer to more accurately and objectively make a decision on if
60fps is necessary, or if they simply need to tighten up their 30fps
game.
Measuring Response Time
Measuring response time is very simple, and consists merely of
videoing the screen and the controller at the same time with a video
camera that records at 60 fps, and then playing this back and counting
the frames between button press and the screen response.
The camera I use is a Canon Powershot SD800IS, a relatively cheap
camera. You need a camera that supports 60 fps (frames per second)
recording. This may be listed as "Sports Mode", or "Fast Frame Rate".
I
believe all the current line of Canon Powershot SD cameras have this
feature. This is a very popular brand of camera, so it is highly likely
that someone on your team will own one.
Set the camera to movie mode, change to to the appropriate mode for
60fps (press the "func" button on the Canon SD and select "Fast Frame
Rate").
Then set it up to point at the television. You don't need to get the
whole screen in shot, just enough so you can see the game actions that
will result from your button presses. Make sure that the controller is
roughly in focus and hold it angled so that the button presses can
clearly be seen.
|
I would love to see this done with MMORPG interfaces. Everyone claims that WoW's interface is the best, and all other games are sluggish compared. I wonder if WoW is just less laggy! I`m sure the stats on that could be pretty interesting :)
Also, Martin, I don't think it will make WoW less laggy because the primary bottleneck for lag in an online game, especially an MMO, is almost always the network connection. Even playing Warcraft or Starcraft, you can notice a slight lag difference between single-player and BNet multiplayer.
Maybe I'm missing something though :) I'm not an expert or anything haha.
In addition, TVs lag differently depending on how much processing is required at different resolutions too. Be sure to set your XMB to the resolution you are testing with (typically 720p)
The best way to measure the response time of your TV is to hook up a PC with DVI/HDMI-outputs in mirror mode, and run a video that has a countdown timer. Use the camera to take snapshots and you'll see the exact number differences between the PC outputting and the TV outputting.
Source : http://hdtvlag.googlepages.com/ourtest
I think the problem is a lot of devs will always sacrifice frame-rate for better visuals because those gains are a lot more obvious to publishers and consumers, it's also fair to say that some games can get away with being more sluggish in terms of response time.
FPS and Driving games (this includes tony hawk:P) where the life or death of a player is heavily focused on the performance of the game so yeah its vital there to keep it tight, however other games have mechanics where that level of continuity is not required and I guess this is where the games start to feel like a bag of nails.
I would like to see devs take a note from the PC world and put in more video options, so for example if your game uses real-time this and real-time that then at least put in the options so "real-gamers" can sacrifice graphics for a smoother gameplay expereince.
you can even make it cheat based so such options are only enabled afte the game has been finished, so you can still have your great looking game upon release:)
Lastly I would say that however you prsent your findings, anyone worth their salt in games should know if a game feels like crap and if they still release agame knowing this, then maybe its best they carry on and hopefully disappear :)
I find it scary that in this gay and age peple are aiming for 30, it should be 60 FPS and if you're dropping frames, then you should be rethinking your level deisgns and art in that area because unless you have crap game engine then it should not be hard for most people to obtain a crisp expereience.
lastly, I don't know how long this will take to filter into the console market, but in the pcmarket it is possible to increase the polling rate of usb devices.
They run at 125hs as standard, but you can overclock to 250, 750, 1000hz, which make the general movement feel a lot smoother.
Same method, just record the button press and the audio output of the game, then look at the difference in the attack portions of the sound.
While I see your point on frames per second being important in a fast twitch game, such as Quake, because they allow for a faster response time you have to take something else into account. YOU are able to run games at 120 fps but many gamers are not. This could, arguably, give you an edge against other players. While this may help you get kills it also kills the spirit of competativness. By keeping games capped at a certain target fps it helps to keep the playing field level by not allowing any one player to leverage their hardware in such a way as to give themselves a better response time. Granted there are other factors, especially things screen resolution, but from a purely "twitch movement" perspective it helps to keep the playing field level.
Sometimes your framerate will be fine but there will be a good half second lag in mouse movement to actual movement on the screen due to the above.
http://www.site-de-bingo.fr
And where a delay is inherent in the move, such as a power attack with a sword, the player still wants SOME response when they press the button, at the very least so they feel the game is responding to them - such as starting an animation.
I think you would be VERY hard pressed to find a game where the developers had DELIBERATELY added a short period of total unresponsiveness after particular button presses. In just about every case the lag is inadvertent, and perhaps unnoticed by the developers. Bringing lag forward as a measurable issue will help ensure developers pay attention to lag, and minimize it where possible.
Even if you accept that in a game such as Heavenly Sword there are particular moves that have additional lag, there is still the issue of the baseline lag - the shortest lag with which the game responds. This lag is often due to underlying technical issues I outlined in another article, and is generally something that is detrimental to the gameplay experience, and something the developers would prefer did not happen.
http://cowboyprogramming.com/2008/05/27/programming-responsiveness/