Other Results
I tested various other games, with various results. I'll list the
results in 60ths of a second, and the results have been adjusted -2 to
account for the plasma lag. All games are on the PS3 unless noted
otherwise, and I've included the PS3 system menus and GTAIV for reference.
Games that run at 60 fps:
- PS3 System menus: 3/60ths
- Guitar Hero III (Xbox 360): 3/60th
- Ridge Racer 7: 4/60ths
- Virtua Tennis 3: 4/60ths
- Ninja Gaiden Sigma: 4/60ths
- PixelJunk Racers: 4/60ths
Games that run at 30 fps:
- Genji: Days of the Blade: 6/60ths
- Tony Hawk's Proving Ground: 8/60ths
- BlackSite: Area 51: 8/60ths
- Halo 3 (Xbox 360) : 8-10/60ths
- Skate: 10/60ths
- GTAIV: 10/60ths
- Harry Potter and the Order of the Phoenix: 10-14/60ths
- Heavenly Sword: 7-18/60ths
The first question that arises is: why are there no PS3 games with a
response time of 3/60ths? The PS3 UI runs at 3/60ths response time, so
it's quite possible. Why do all the 60fps games run at 4/60ths?
Still,
that's still pretty good, and those games all feel quite responsive.
Guitar Hero III on the Xbox 360 manages a very impressive 3/60ths - very
important for that type of game, especially when it's actually 5/60ths
on many TVs.
Next, why so much variance between the 30fps games - ranging from 6/60ths to 18/60ths?
It's worth noting that while EA's Skate seems like it would be a bit
sluggish at 10/60ths, it actually feels quite responsive, in large part
due to its use of stick gestures for input, as the movement starts
before the gesture is complete, yet the player mentally synchronizes it
with the end of the gesture, and so it actually seems very responsive.
Then what about the games where the lag varies? Harry Potter is
10-14 which is bad enough, but Heavenly Sword is an astonishing 7-18.
It takes 7/60ths to start an attack, but 18/60ths to start to turn
around (See movie: mvi_4263w).
Clearly something is wrong there. I would consider that a bug.
It's sad
that the programmer made the effort that allowed for a 7/60th response
time, but then someone else messed up down the line, making the turn
take nearly a third of a second. Halo 3 is another example, with the
shooting and moving being 8, but the jumping being 10.
Particularly
interesting here in the 30fps category is Genji: Days of the Blade.
This is a very similar game to Ninja Gaiden, and yet Genji runs at 30,
while Ninja Gaiden runs at 60.
However, the fact that Genji runs at 30
is barely noticeable, and does not detract from the game at all. In
part this is because of the way the camera moves smoothly through the
scene with very few rapid pans. It's also because of the low contrast
graphics and motion blur. But it's also because Genji's response rate
is 6/60ths, very similar to Ninja Gaiden's 4/60ths.
Ninja
Gaiden is a faster paced game than Genji, with the main character
jumping around rapidly, and the camera tightly following him - so the
graphical benefits of 60fps are more apparent. The low response time
feels good as well.
However, since Genji also has a low response time,
it would benefit very little from running at 60 fps. As it runs at
30fps, this gives the designers the opportunity to put more graphics,
enemies and special effects on screen, and reduces the pressure on
programmers and artists to constantly strive to maintain 60, which can
be a difficult factor in development.
Conclusions and Suggestions
Games that run at 60fps all seem to have a response time to 4/60ths, and while 3/60ths is possible, 4/60ths is a very good response time.
Some games running at 30fps have a response time of 8/60ths or 10/60ths (and some peak even higher).
Genji shows us that a response time of 6/60ths is possible while
running at 30 fps. 10/60ths can be too long, especially when combined
with the processing delays in flat panel TVs which can push it up to
12/60ths. 1/5th of a second (200ms) is too long to wait for a gun to
fire, and introduces annoying sluggishness when moving around or
steering a car.
Some games have an inconsistent response time. Heavenly
Sword varies from 7 to 18. If the system is capable of 7, then all
moves should start in 7. Developers should verify ALL their response
times, as other factors, such as animation, might be creating lag in
specific places.
I suggest that game developers use this simple technique to measure the response time in their games,
at least to verify that their assumptions are correct. If they are
running at 60fps, then they should not be above 4/60ths. If they are
running at 30fps, then they strive to duplicate Genji's responsiveness
of 6/60ths, and certainly not slip below 8/60ths. I suggest that they
keep in mind that flat panel TVs (which are probably a majority in
gamers households, and certainly in game reviewers' households) add an
additional 2 frames of lag, which makes it EVEN MORE important to keep
programmed lag to a minimum.
I also suggest that game reviewers begin to use this technique to measure lag,
and to include the measurement of lag in their reviews. While the
subjective views of the reviewer are important and valuable, an
objective measurement of response time would be a very useful
additional piece of information for the person considering buying the
game.
Placing this information in a game review would encourage
developers to produce more responsive games, which benefits everyone.
|
Printer-Friendly Version
I would love to see this done with MMORPG interfaces. Everyone claims that WoW's interface is the best, and all other games are sluggish compared. I wonder if WoW is just less laggy! I`m sure the stats on that could be pretty interesting :)
Also, Martin, I don't think it will make WoW less laggy because the primary bottleneck for lag in an online game, especially an MMO, is almost always the network connection. Even playing Warcraft or Starcraft, you can notice a slight lag difference between single-player and BNet multiplayer.
Maybe I'm missing something though :) I'm not an expert or anything haha.
In addition, TVs lag differently depending on how much processing is required at different resolutions too. Be sure to set your XMB to the resolution you are testing with (typically 720p)
The best way to measure the response time of your TV is to hook up a PC with DVI/HDMI-outputs in mirror mode, and run a video that has a countdown timer. Use the camera to take snapshots and you'll see the exact number differences between the PC outputting and the TV outputting.
Source : http://hdtvlag.googlepages.com/ourtest
I think the problem is a lot of devs will always sacrifice frame-rate for better visuals because those gains are a lot more obvious to publishers and consumers, it's also fair to say that some games can get away with being more sluggish in terms of response time.
FPS and Driving games (this includes tony hawk:P) where the life or death of a player is heavily focused on the performance of the game so yeah its vital there to keep it tight, however other games have mechanics where that level of continuity is not required and I guess this is where the games start to feel like a bag of nails.
I would like to see devs take a note from the PC world and put in more video options, so for example if your game uses real-time this and real-time that then at least put in the options so "real-gamers" can sacrifice graphics for a smoother gameplay expereince.
you can even make it cheat based so such options are only enabled afte the game has been finished, so you can still have your great looking game upon release:)
Lastly I would say that however you prsent your findings, anyone worth their salt in games should know if a game feels like crap and if they still release agame knowing this, then maybe its best they carry on and hopefully disappear :)
I find it scary that in this gay and age peple are aiming for 30, it should be 60 FPS and if you're dropping frames, then you should be rethinking your level deisgns and art in that area because unless you have crap game engine then it should not be hard for most people to obtain a crisp expereience.
lastly, I don't know how long this will take to filter into the console market, but in the pcmarket it is possible to increase the polling rate of usb devices.
They run at 125hs as standard, but you can overclock to 250, 750, 1000hz, which make the general movement feel a lot smoother.
Same method, just record the button press and the audio output of the game, then look at the difference in the attack portions of the sound.
While I see your point on frames per second being important in a fast twitch game, such as Quake, because they allow for a faster response time you have to take something else into account. YOU are able to run games at 120 fps but many gamers are not. This could, arguably, give you an edge against other players. While this may help you get kills it also kills the spirit of competativness. By keeping games capped at a certain target fps it helps to keep the playing field level by not allowing any one player to leverage their hardware in such a way as to give themselves a better response time. Granted there are other factors, especially things screen resolution, but from a purely "twitch movement" perspective it helps to keep the playing field level.
Sometimes your framerate will be fine but there will be a good half second lag in mouse movement to actual movement on the screen due to the above.
http://www.site-de-bingo.fr
And where a delay is inherent in the move, such as a power attack with a sword, the player still wants SOME response when they press the button, at the very least so they feel the game is responding to them - such as starting an animation.
I think you would be VERY hard pressed to find a game where the developers had DELIBERATELY added a short period of total unresponsiveness after particular button presses. In just about every case the lag is inadvertent, and perhaps unnoticed by the developers. Bringing lag forward as a measurable issue will help ensure developers pay attention to lag, and minimize it where possible.
Even if you accept that in a game such as Heavenly Sword there are particular moves that have additional lag, there is still the issue of the baseline lag - the shortest lag with which the game responds. This lag is often due to underlying technical issues I outlined in another article, and is generally something that is detrimental to the gameplay experience, and something the developers would prefer did not happen.
http://cowboyprogramming.com/2008/05/27/programming-responsiveness/