3D GameGauge 2.0

We took to heart the lessons we learned with 3D GameGauge 1.0. We worked with game developers and graphics card vendors to help define 3D GameGauge 2.0. (You can download the full spec in MS Word format).

• 3D GameGauge 2.0 requires a minimum of 1024x768x16 as the maximum resolution. Running at a higher resolution minimizes the impact of the CPU.
• There can be no frame rate limiter.
• We wanted to see triple-buffering used, because testing with flip-on-vsync off is a pain. Triple-buffering eliminates the harmonic effect prevalent with double-buffering, but you still can't run faster than the refresh rate. Unfortunately, we've discovered with some of the newer cards that they can peg the frame rate at 85Hz. So it looks like we'll still have to test with flip-on-vsync disabled. It's not very useful if we tell people, "All cards run at 60Hz, so just pick one."
• Ideally, we'd like the AI turned off during playback. This minimizes variations in the playback. It wasn't possible in all cases. One of our titles is Jane's WWII Fighters. The playback engine still has the AI active. I put in a lot of time handcrafting a WWII Fighters cinema that would minimize the impact of the AI. Even now, if you see a plane crash in the first fifteen seconds, you need to discard that test.

So what is 3D GameGauge 2.0? On the surface, it's a series of seven titles that we run to test 3D graphics performance on graphics cards and systems. But it's really a spec that any game developer can use to build in a standard performance test into their system. The entire spec is available at the end of this article, but here's a summary.

Base Requirements

• The title must run at 1024x768x16 or higher.
• The minimum, maximum and average frame rate is calculated, and written to a text file called FPS.TXT in the game's working directory. (Do the min/max test once per second, but not more often). The format of fps.txt looks like:

87.04 FooBar v1.01

5 Min

100 Max

• The title must be able to be run from a command line, with appropriate parameters (resolution, render states, color depth, and so on) passed in. This allows easier batch automation.
• The game must not have a frame rate limiter.
• The game must support either OpenGL or Direct3D or both.
• The playback must generate at least 2,000 frames to enable better reproducibility.
• The demo should include some real action, not just a flythrough of the world.

Optional, High Wants

• When fps.txt is created, also write to it the running frame rate that was calculated once per second, so a histogram can be generated. The format would look like:

87.04 FooBar v1.01

5 Min

100 Max

91 Second 1

75 Second 2

102 Second 3

...

.68 Second 305

• True color rendering support (32bpp)
• 32-bit source artwork (textures)
• Textures with resolutions higher than 256x256
• Resolutions up to 1600x1200 in all color depths

3D GameGauge 2.0 Titles

If you build this capability into your title, it won't necessarily become part of CGW's GameGauge tests. We need to keep the test suite fixed, for some period of time, so we can do historical comparisons. (We’ll be doing a new version in the fall, to 2.1, so if you do want your game included, contact me at loyd@uncertainty.com.) However, it does give your users a useful tool, and you may see your title all over the Internet, if it's a good one. Here's the current list of confirmed titles.

Jane's WWII Fighters (Electronic Arts) - Direct3D

Madden 99 (EA Sports) - Direct3D

Expendable (Rage) - Direct3D

Descent III (Interplay / Outrage) - Direct3D

Half-Life (Valve) - OpenGL

Powerslide (Ratbag) - Direct3D

We're also very close to having Unreal. However, Epic is slaving away mightily at getting Unreal Tournament done, and we're sympathetic to their schedule requirements. The Unreal engine will be used in quite a variety of games in the future. If they don't make it this time, it will be in the fall refresh.

As you can see, the genre spread is wider than GameGauge 2.0. It would be good to have a 3D-accelerated strategy game (such as Myth 2 or the upcoming Shogun) added to the list, but overall, we're quite pleased.

It's BenchMARKETING

Even if you're not included in CGW's GameGauge suite, making it easier for your customers to test the performance of your title on their system really helps them make better buying decisions. I get e-mail daily from people asking "Which 3D card should I get?"

The question doesn't have a simple answer, because it all depends on what games they like to play. If you build 3D GameGauge into your title, you'll be helping them out. And there are a lot of web sites which aren't affiliated with Ziff-Davis out there doing testing, and that can give your title exposure as well. All in all, it's a small modification that can generate a lot of exposure.

Loyd Case is a freelance writer and frequent contributor to Computer Gaming World (print) and Gamespot (web). He spends altogether too much time up to his elbows inside computers trying to get yet another new graphics card working. His free time is spent downhill skiing, raising two daughters and hosting LAN parties in his basement office. He can be reached at loyd@uncertainty.com.