Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
April 19, 2014
arrowPress Releases
April 19, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM TechWeb sites:


Metacritic Begins Aggregating Individual Developer Scores
Metacritic Begins Aggregating Individual Developer Scores
March 25, 2011 | By Leigh Alexander

March 25, 2011 | By Leigh Alexander
Comments
    35 comments
More: Console/PC, Smartphone/Tablet, Indie, Business/Marketing



Review score aggregator Metacritic has begun offering listings for individual game developers, providing something of a profile page with an aggregate score for all the work on which one has been credited.

Although aggregates for the work of prominent developers, like Ken Levine, Peter Molyneux and Cliff Bleszinski (89, 82 and 86, respectively) are available, all developers with credits on Metacritic-ranked games can find their profile through the first name-hyphen-last name format in the URL.

The data pulls from GameFAQs, with which Metacritic shares a parent. As such, info is far from complete; numerous industry veterans have told Gamasutra that their Metacritic page represents only a partial profile of games they've shipped.

Under its frequently-asked questions, Metacritic recommends that developers submit additions and changes in their crediting directly to GameFAQs.

A developer's Metacritic profile shows his or her highest-scored title, his or her lowest-scored title, and an average score, along with a list of the individual games on which they've worked and the role they played therein.

"Man, it's bad enough games are judged by Metacritic," quips one multiplayer designer with six years in the industry anonymously to Gamasutra. "Now I'm going to be, too?"

Metacritic has ranked companies as a whole for some time; in 2010 Take-Two was the company with the highest overall portfolio score.


Related Jobs

Treyarch / Activision
Treyarch / Activision — Santa Monica, California, United States
[04.19.14]

Associate Art Director - Treyarch
Treyarch / Activision
Treyarch / Activision — Santa Monica, California, United States
[04.19.14]

Associate Animator (temporary) - Treyarch
Activision Publishing
Activision Publishing — Vancouver, British Columbia, Canada
[04.19.14]

Principal Graphics Programmer
Activision Publishing
Activision Publishing — Santa Monica, California, United States
[04.19.14]

Executive Producer-Skylanders










Comments


Rob Wright
profile image
This is just hilarious. If only people know how little work these big name developers actually put into some of these titles that carry their names...

JB Vorderkunz
profile image
regardless the 'real' breakdown by project or studio, it does seem like this is a very uncritical way to 'rank' devs...

Rob Wright
profile image
I'm just curious what Metacritic will give Sid Meier for Civilization V.....

Tim Carter
profile image
It's isn't the quantity of work that counts.



It's the quality. And the central importance.

Sting Newman
profile image
They both matter.

Tim Carter
profile image
Well Miles Copeland put it this way. When you buy a concert ticket to see The Police, if you find out that the sound engineer or the lighting tech was replaced with a new guy, you'll probably still go. But if you find out that Sting or Stewart Copeland have been replaced with a new guy, you'll probably stay at home.



So quality matters far more.

Chris Remo
profile image
The credits are also incomplete, which makes the ratings inaccurate (inaccurate according to the intended formula, that is; it's clear they represent no meaningful accuracy in reality). They reportedly draw from the GameFAQs database, which has huge gaps in terms of both game entries and credits. Even some games with credits are missing many of the important credits--I've seen several games that only credit the creative director and a huge list of voice actors, but no other actual developers on the project.

Michael Krehan
profile image
What a bad choice. Why GameFAQ? Why not MobyGames, where the approvers at least ask for a scan of the inlay to verify the list? It may not have all recent titles on file, but at least you can count more on the veracity and completeness of those that are listed.

Craig Timpany
profile image
Even MobyGames is still pretty horrible. No way for the producer to say "hey, here's a CSV of the credits list that went into the GM". It's so very very time consuming and error prone.



Also, the corrections process hardly ever works. Maybe if they fixed that, they wouldn't have to be so pedantic about credits scans and screenshots.

Craig Timpany
profile image
(Not that MetaCritic would ever use data from MobyGames anyway, it's part of GameFly media, not CNet)

Evan Bell
profile image
So if you have a high score you put that in your CV. If you don't then you hope no one bothers to look. What is next? Un-moderated user reviews for devs?

Megan Fox
profile image
So it seems.



Were you a programmer on a game with great tech, that flopped at market because of improper release schedule management or a screw-up in marketing? TOO BAD! You're still a 56!



Hmmm.

Jacek Wesolowski
profile image
Consider it a good calibration tool. Would you like to work for someone who makes hiring decisions based on a simple stat? Would you hire someone who puts their "score" in their CV, because they don't realise it's meaningless?



Though it is sad that this is probably going to incite some of the stupidest fanboy wars ever.

Kamruz Moslemi
profile image
Oh my god, this is a travesty, I go over to Shinji Mikami's page and wonder why the guy that made one of the greatest masterpieces of the 6th generation only had a score of 80. Then I scroll down to see that there are a lot of bad scores among his titles, and they are things like the GameCube port of Resident Evil 2 (59), the PC port of Dino Crisis (59) and the like.



Really? Really!? This is laughable.

Rob Wright
profile image
Again, it boggles the mind: of all the "art" criticism in popular culture, the reviews with the worst quality and most screwed up metrics somehow have the most power and influence over their medium. As a friend of mine likes to say, it's a goddamn clown show....

Paul Wrider
profile image
Considering the number of titles I've worked on that were canceled/never credited, is there a way for me to choose to be de-listed? lol

Jeremy Alessi
profile image
It's like baseball cards for game developers. That settles the art debate, we're obviously a sport now.

Tim Carter
profile image
In the vast history of western art, the focus has always been on key individuals. You don't know the guys who erected the scaffolding for the Sistine Chapel's ceiling. But you do know Michelangelo.

Wendy Jones
profile image
This definitely doesn't bode well for the game industry. Metacritic is famous for scores that a lot of times don't represent the quality of the game they measure. I definitely don't want to be measured as a developer by this system. I've seen companies get caught up in designing to metacritic scores instead of their fan base, I wouldn't want their hiring practice to go the same way.



As is stands, both GameFaq and Moby Games are inadequate databases of credits for developers. If we are to be judged, it should at least be with complete and verifiable data.

Daniel Gooding
profile image
In my opinion, in some ways it really feels like Metacritic isn't something that casual developers should fear, but people targeting game developers, or extreme gamers should loathe.



Honestly I had never heard of it till I started hanging out with game developers maybe 4-5 years ago, even now I don't use it, I just look for gameplay trailers.



I've asked dozens of people if they have heard of it, and the only people that had ever used it before were mainly game developers. Students who want to be game developers, or broke guys, who trusted rating sites.



My brother, and sister who buy games for themselves, and their kids don't have any idea what it is. My best friend who despises the internet, but plays games all the time never heard of it, and buys a bunch of games sometimes still just based on the coverart of the packaging. (which I think is crazy, but awesomely simplistic of him)

I've asked kids, and teens passing by in gamestops, and other venues how they choose their games. Most of the time it's word of mouth, or magazines.



Perhaps it is a different story elsewhere, but here in the midwest, few people know metacritic, and yes people still buy games.

Bob Stevens
profile image
Guys with 25 years in the industry and hundreds of titles under their belt are ending up on here with 1 game developed and scores of like 20%... if they're on there at all. Mobygames has been collecting credits info for like 10 years and is still nowhere near complete or accurate. I expect the GameFAQs/Metacritic credits DB to have similar levels of success when it comes to completeness and accuracy 10 years from now.



Doesn't matter, anyone daft enough to use this stuff for hiring isn't worth working for (or hiring). It's a cute, fun, meaningless, and inaccurate statistic.

Tim Carter
profile image
Perhaps. But at least it's a step in the right direction: recognition of individual creators.



They have the best chance of forwarding the needs of creative. Otherwise, we will remain where we are now: the most powerful individuals in this industry being suits.

Noah Falstein
profile image
I've been in the industry for 31 years, worked on hundreds of titles, and they have a grand total of one score from a game I consulted on for a single week. That's one week in about 1618. Not exactly stunning accuracy.

Kale Menges
profile image
The ratings for games are never broken down into categories such as "Programming Efficiency", "Artistic Proficiency" or "Pipeline Efficiency" or "Developer Creativity vs. Publisher Demands", so how the hell do you legitimately acquire public data to create performance statistics for specific individuals supposedly involved in the creation of a game in order to somehow "rank" them against other individual developers? I've played plenty of ugly looking games that had fantastic game and level design, likewise plenty of beautifully polished looking games that were terrible to play, so the performance of a final product is impossible to use as any kind of valid metric for guaging the contributions of any one individual involved in a game's development. I'm sorry if I'm preaching to the choir on this, but this whole idea is just plain ludicrous and insulting to all professional and independent game developers alike.

Wolf Wozniak
profile image
I like how people are actually debating this.



Here's what everyone should unanimously be saying...



"This is fucked up."

Daniel Carvalho
profile image
Hahaha, exactly. There should be no argument here. We all know the list is retarded at best.

Dean Ferguson
profile image
Can see Alan Smithy getting a lot of credits from here on out.

Ben Droste
profile image
I'd much rather people judge my work from my portfolio as opposed to some arbitrary number, so it's probably a good thing I'm not listed on there. Not that I expect potential employers would be silly enough to that, or if they are I don't think I'd be too confident working for them...



But the real problem I have with it is this: how can they assign a score, based on average scores of the games I've worked on, to judge my 'score' as an environment artist? What has the overall score of a game, let down or bumped up by any number of things outside of my control got to do with my work? *Maybe* if it were an aggregate of graphics scores it might have some slight merit, but even that's a stretch.



I guess what I'm saying is: what do they think they are actually rating??

Alan Youngblood
profile image
We were tossing around this really cool idea at GDC: Take classic works of literature, paintings, sculptures, architecture and review them meta-critic style so everyone can see if they want to buy or view them in a museum based on a numerical score. (Note the tongue-in-cheek sarcasm here).



Van Gogh's Starry Night: 67% This piece has good colors but it looks like something my kindergartener made finger painting today in class.



Pisano's Leaning Tower of Pisa: 43% Ok, first off did Pisano really make this thing? I mean someone else is taking credit for it. Get an IP lawyer and sue him so we can be sure our facts are straight. Now about the tower: this is a pretty shoddy piece of architecture. I mean it doesn't even stand up straight. I would not advise buying a plane ticket to Tuscany Italy just to see this. Let's hope the Tower of Pisa 2 gets rid of the "gangsta lean" and works as a real free standing structure. I mean Stonehenge was made sturdier and it was built by like, cavemen or something.

Rob Wright
profile image
Awesome idea!



On a serious note, has anyone noticed that on MetaCritic any movie score above 60 is in the "green" zone, yet for games to get into the "green" they have to top 75? Isn't that a little odd, since MC is basically acknowledging a double standard?



You can play a pretty fun game on MC by taking a movie score and seeing what its counterpart is in the game review section, and vice versa. Source Code would be Homefront, Rango would Hard Corps: Uprising, and Blue Valentine would be Blur.

Huck Terrister
profile image
Hmm. Always knew this "Richard Garriott" fellow was overrated. Only a 66? Sure, SOME magazines will claim that that is "above average" but now I guess we all know he'll never make it in this business



http://www.metacritic.com/person/richard-garriott

Matthew Mouras
profile image
@Wesley Baker: Ha! There's solid proof it's meaningless.



This is silly - just another product of a brave new information world obsessed with metrics. It reminds me of the public "ranking" of teachers in California and what a stink that created. Teachers didn't like it either and for good reason. It's a misrepresentation.



Though I guess the exact problem isn't that flawed ranking systems like this exist, the problem happens when people start assigning any significance to them.

Tore Slinning
profile image
David Gaider is then rated higher then Tim Cain?



Thats just wrong...

Jason Bay
profile image
Seems like this might just be a clever way to motivate developers to help them complete the GameFAQs credits database. Crowdsourcing FTW?

Matt Cratty
profile image
They have me down for having developed one game.



I don't have the resume that many of you do, but these sites are pretty useless if that's the best they can do.


none
 
Comment: