Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
August 3, 2020
arrowPress Releases







If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Survey: Game Score-to-Sale Theory Again Disproven

Survey: Game Score-to-Sale Theory Again Disproven

September 19, 2006 | By Brandon Boyer

September 19, 2006 | By Brandon Boyer
Comments
    Post A Comment
More: Console/PC



In the new issue of the Susquehanna Financial Group's Video Game Journal, authors Jason Kraft and Chris Kwak follow up on their earlier study, and again statistically disprove a direct correlation between video game review scores and the resulting performance at retail, reiterating that there is little relationship between a game's critical and commercial reception.

The original study, conducted in December of last year, took a sample of 275 games across genres for the PlayStation 2 and charted their relative NPD sales data and Metacritic scores, concluding, "The low correlation may indicate that this random sample, which includes PS2 titles across six years and multiple genres, is the result of a failure to account for a number of other factors. Ratings may only tell part of the story."

Saying that the study "touched a nerve" and caused a flood of emails and calls with complaints that the authors' "sample size was too small; others that we had not looked at multi-variable regressions; and still others that we did not disclose enough of the data," Susquehanna followed up with a wider range of samples.

This time taking a sample of 1200 PlayStation 2 titles, the authors once again charted relative NPD and Metacritic data, and once again have concluded "the correlation between game ratings (Metascores) and unit sales is statistically insignificant."

In fact, the study found that by increasing the sample, "only 15.8% of the movement in game unit sales ... can be explained by movements in game ratings," a figure down from its original 17.3% from the sample of 275 games.

To help try and trap for other factors that might explain the findings, the authors looked specifically at franchise correlation, including sequels and new franchises, and selected data from other consoles, in all cases failing to find sufficient direct score to sales matches outside extreme isolated cases.

In conclusion, the authors state, "a theory (that game ratings matter) that fails under scrutiny is accepted as conventional wisdom. Conventional wisdom is wrong. And we have not even addressed the causation argument - that a higher rating causes a game to sell better. There is no reason to argue causation, because while correlation does not equal causation, the absence of correlation means no causation in our case."


Related Jobs

Mountaintop Studios
Mountaintop Studios — Los Angeles, California, United States
[08.03.20]

Engine/Systems Engineer (remote)
Mountaintop Studios
Mountaintop Studios — Los Angeles, California, United States
[08.03.20]

Graphics Engineer (remote)
Yacht Club Games
Yacht Club Games — Los Angeles, California, United States
[08.03.20]

Senior 3D Technical Artist
Mountaintop Studios
Mountaintop Studios — Los Angeles, California, United States
[08.03.20]

Network Engineer (remote)









Loading Comments

loader image