3D GameGauge 1.0

Unfortunately, a lot of the game performance testing that's done out there isn't always done in a rigorous fashion, so it becomes difficult to compare results. Even if you ignore system differences (all Pentium III 500MHz machines are not necessarily equal), there are problems. If one user runs Quake II timedemos using version 3.14, how does that compare to 3.20? The short answer is, it doesn't.

At CGW, we wanted a standard way of determining performance of 3D graphics cards or systems using real games. So we came up with the idea of 3D GameGauge.

3D GameGauge is really very simple. The basic concept was to have fixed length demo loops that would just test the rendering capability of the 3D card. However, we did want to have audio active -- after all, few of us play with the sound turned off. With some early graphics cards, we saw performance actually decrease substantially when turning on audio. A 3D accelerator needs to be a good "systems citizen," after all.

3D GameGauge 1.0 consisted of six titles: Forsaken (a prerelease demo), Incoming, Turok, Quake II, GL Quake and F22 Air Dominance Fighter. We began using it in early 1998. As with any 1.0 product, there were some problems:

• The early demo of Forsaken was flaky. It didn't like some forms of audio acceleration, and there was a weird bug in the way it related to Windows 98 that would sometimes cause severe paging to virtual memory for no good reason.
• F22 ADF was frame-rate limited to 50fps. Early last year, that wasn't a big problem, but it became one later.
• F22 ADF's playback engine was limited to 640x400, despite DID's insistence that it was really running at 800x600.
• There was no real front end to launch all the titles, making it a manual -- and hence, tedious -- process.
• The whole test was pretty first-person shooter intensive. There wasn't a good genre spread.
• In retrospect, using the sum of the frame rates as the overall score wasn't a good scoring methodology. Forsaken would generate huge scores, and minimize the impact of other titles.