Calibration and Measuring
Plasma and LCD televisions introduce lag of their own, as they
frequently do some additional processing to a picture before they
display it. This can be as much as a few 60ths of a second, which can
be quite significant. Ideally you would do your testing on a CRT
Television, which should have no additional lag (the cheaper the
better). Your flat panel TV might also have a "game mode" which will
reduce (but not eliminate) the lag.
To find how much lag your TV adds, you need to compare an easily
measured response with the same response on a CRT TV, or with a known
value. To simplify things for you I've measured the response time of
the PS3 system menus, which come out at a very solid 3/60ths on a CRT,
and 5/60th on the plasma TV shown above.
Hence I need to subtract
2/60th of a frame to get an accurate measure of responsiveness when
using the plasma for measurements. Make the appropriate adjustment for
your TV.
This additional lag, however, should not be ignored, and it
should be taken into account when developing the game. Many people will
run your game on a TV that introduces additional lag. Steps should be
taken to minimize the negative effects this unavoidable lag can have.
To measure the response time (lag) the simplest tool I found has been the free Quicktime viewer.
With Quicktime installed, right click on the video file from your
camera, and then quick on "Open with Quicktime". You can then use the
arrow keys to frame advance 1/60th at a time.
For a particular response time, navigate the video to the first
frame that shows the button fully pressed, and then count additional
frames from there until you see the first indication of a response on
screen. The number of frames you counted is your response time in
1/60ths of a second. You can convert to milliseconds by dividing by 60
and multiplying by 1000 (12/60ths is 200 ms)
Here are the frames for the PS3 system menu on the CRT TV:
In this first image (above), the thumb is descending to the "down"
button on the D-Pad. It's important to press the buttons with the thumb
starting off the pad and use a rapid hitting motion, so you can see the
precise frame when the button is fully pressed. Note the slight motion
blur on the thumb - this is also a useful indicator of if it is still
moving or not.
Now the thumb is clearly stopped and is pressing the button. Since
this is the first frame where we see the button pressed, then we start
counting from here, at zero (if the response were visible here, you'd
have zero lag, impossible with current systems).
Now (above) we are on the first frame of actual lag, note the same line on the menu is still highlighted.
Second frame of lag, still no change.
Third frame. The menu begins to move. This means that the PS3 system
menu has a response time of 3/60ths of a second, or 50 ms
(milliseconds). This is VERY GOOD, and is the fastest you can
realistically expect from the PS3.
While it's theoretically possible to
go to 2/60ths, this has severe performance problems. I don't think
1/60th is possible on the PS3. So - all games running at 60 fps should
shoot for 3/60ths as an ideal response time. This also means games
running at 30 fps should shoot for 6/60ths. That's in a perfect world,
and as we shall see, actual results vary.
Results for Grand Theft Auto IV
The first thing I tried was shooting a gun in Grand Theft Auto IV for the PS3 on
a plasma television. This is done with the right trigger button (R2),
which feels (slightly) like pulling the trigger of a real gun.
In real
life, there is zero delay between the trigger reaching the end of its
travel, and the bullet leaving the gun. In GTAIV, this delay is
somewhat longer. Here's the actual movie, so you can frame advance it yourself, and here are the relevant frames:
Note on the first frame (-1) , the finger is still moving, we start
counting on the next frame (0) when the finger is fully on the button
and the button is fully depressed. We then count until the first
response, which comes at frame 12.
This indicates a response time of
12/60ths. Since it's measured on the plasma TV, we adjust this to
10/60ths. This gives us a raw response time for GTAIV of 166 ms (200
ms on flat panel TVs).
I repeated this test for turning and jumping, and got the same
results. This is rather a long response time, and correlates with
people reporting the game being sluggish and unresponsive. The delay in
firing the gun after pulling the trigger is quite noticeable.
|
Printer-Friendly Version
I would love to see this done with MMORPG interfaces. Everyone claims that WoW's interface is the best, and all other games are sluggish compared. I wonder if WoW is just less laggy! I`m sure the stats on that could be pretty interesting :)
Also, Martin, I don't think it will make WoW less laggy because the primary bottleneck for lag in an online game, especially an MMO, is almost always the network connection. Even playing Warcraft or Starcraft, you can notice a slight lag difference between single-player and BNet multiplayer.
Maybe I'm missing something though :) I'm not an expert or anything haha.
In addition, TVs lag differently depending on how much processing is required at different resolutions too. Be sure to set your XMB to the resolution you are testing with (typically 720p)
The best way to measure the response time of your TV is to hook up a PC with DVI/HDMI-outputs in mirror mode, and run a video that has a countdown timer. Use the camera to take snapshots and you'll see the exact number differences between the PC outputting and the TV outputting.
Source : http://hdtvlag.googlepages.com/ourtest
I think the problem is a lot of devs will always sacrifice frame-rate for better visuals because those gains are a lot more obvious to publishers and consumers, it's also fair to say that some games can get away with being more sluggish in terms of response time.
FPS and Driving games (this includes tony hawk:P) where the life or death of a player is heavily focused on the performance of the game so yeah its vital there to keep it tight, however other games have mechanics where that level of continuity is not required and I guess this is where the games start to feel like a bag of nails.
I would like to see devs take a note from the PC world and put in more video options, so for example if your game uses real-time this and real-time that then at least put in the options so "real-gamers" can sacrifice graphics for a smoother gameplay expereince.
you can even make it cheat based so such options are only enabled afte the game has been finished, so you can still have your great looking game upon release:)
Lastly I would say that however you prsent your findings, anyone worth their salt in games should know if a game feels like crap and if they still release agame knowing this, then maybe its best they carry on and hopefully disappear :)
I find it scary that in this gay and age peple are aiming for 30, it should be 60 FPS and if you're dropping frames, then you should be rethinking your level deisgns and art in that area because unless you have crap game engine then it should not be hard for most people to obtain a crisp expereience.
lastly, I don't know how long this will take to filter into the console market, but in the pcmarket it is possible to increase the polling rate of usb devices.
They run at 125hs as standard, but you can overclock to 250, 750, 1000hz, which make the general movement feel a lot smoother.
Same method, just record the button press and the audio output of the game, then look at the difference in the attack portions of the sound.
While I see your point on frames per second being important in a fast twitch game, such as Quake, because they allow for a faster response time you have to take something else into account. YOU are able to run games at 120 fps but many gamers are not. This could, arguably, give you an edge against other players. While this may help you get kills it also kills the spirit of competativness. By keeping games capped at a certain target fps it helps to keep the playing field level by not allowing any one player to leverage their hardware in such a way as to give themselves a better response time. Granted there are other factors, especially things screen resolution, but from a purely "twitch movement" perspective it helps to keep the playing field level.
Sometimes your framerate will be fine but there will be a good half second lag in mouse movement to actual movement on the screen due to the above.
http://www.site-de-bingo.fr
And where a delay is inherent in the move, such as a power attack with a sword, the player still wants SOME response when they press the button, at the very least so they feel the game is responding to them - such as starting an animation.
I think you would be VERY hard pressed to find a game where the developers had DELIBERATELY added a short period of total unresponsiveness after particular button presses. In just about every case the lag is inadvertent, and perhaps unnoticed by the developers. Bringing lag forward as a measurable issue will help ensure developers pay attention to lag, and minimize it where possible.
Even if you accept that in a game such as Heavenly Sword there are particular moves that have additional lag, there is still the issue of the baseline lag - the shortest lag with which the game responds. This lag is often due to underlying technical issues I outlined in another article, and is generally something that is detrimental to the gameplay experience, and something the developers would prefer did not happen.
http://cowboyprogramming.com/2008/05/27/programming-responsiveness/