Review score aggregator Metacritic has begun offering listings for individual game developers, providing something of a profile page with an aggregate score for all the work on which one has been credited.
Although aggregates for the work of prominent developers, like Ken Levine, Peter Molyneux and Cliff Bleszinski (89, 82 and 86, respectively) are available, all developers with credits on Metacritic-ranked games can find their profile through the first name-hyphen-last name format in the URL.
The data pulls from GameFAQs, with which Metacritic shares a parent. As such, info is far from complete; numerous industry veterans have told Gamasutra that their Metacritic page represents only a partial profile of games they've shipped.
Under its frequently-asked questions, Metacritic recommends that developers submit additions and changes in their crediting directly to GameFAQs.
A developer's Metacritic profile shows his or her highest-scored title, his or her lowest-scored title, and an average score, along with a list of the individual games on which they've worked and the role they played therein.
"Man, it's bad enough games are judged by Metacritic," quips one multiplayer designer with six years in the industry anonymously to Gamasutra. "Now I'm going to be, too?"
Well Miles Copeland put it this way. When you buy a concert ticket to see The Police, if you find out that the sound engineer or the lighting tech was replaced with a new guy, you'll probably still go. But if you find out that Sting or Stewart Copeland have been replaced with a new guy, you'll probably stay at home.
The credits are also incomplete, which makes the ratings inaccurate (inaccurate according to the intended formula, that is; it's clear they represent no meaningful accuracy in reality). They reportedly draw from the GameFAQs database, which has huge gaps in terms of both game entries and credits. Even some games with credits are missing many of the important credits--I've seen several games that only credit the creative director and a huge list of voice actors, but no other actual developers on the project.
What a bad choice. Why GameFAQ? Why not MobyGames, where the approvers at least ask for a scan of the inlay to verify the list? It may not have all recent titles on file, but at least you can count more on the veracity and completeness of those that are listed.
Even MobyGames is still pretty horrible. No way for the producer to say "hey, here's a CSV of the credits list that went into the GM". It's so very very time consuming and error prone.
Also, the corrections process hardly ever works. Maybe if they fixed that, they wouldn't have to be so pedantic about credits scans and screenshots.
So if you have a high score you put that in your CV. If you don't then you hope no one bothers to look. What is next? Un-moderated user reviews for devs?
Were you a programmer on a game with great tech, that flopped at market because of improper release schedule management or a screw-up in marketing? TOO BAD! You're still a 56!
Consider it a good calibration tool. Would you like to work for someone who makes hiring decisions based on a simple stat? Would you hire someone who puts their "score" in their CV, because they don't realise it's meaningless?
Though it is sad that this is probably going to incite some of the stupidest fanboy wars ever.
Oh my god, this is a travesty, I go over to Shinji Mikami's page and wonder why the guy that made one of the greatest masterpieces of the 6th generation only had a score of 80. Then I scroll down to see that there are a lot of bad scores among his titles, and they are things like the GameCube port of Resident Evil 2 (59), the PC port of Dino Crisis (59) and the like.
Again, it boggles the mind: of all the "art" criticism in popular culture, the reviews with the worst quality and most screwed up metrics somehow have the most power and influence over their medium. As a friend of mine likes to say, it's a goddamn clown show....
In the vast history of western art, the focus has always been on key individuals. You don't know the guys who erected the scaffolding for the Sistine Chapel's ceiling. But you do know Michelangelo.
This definitely doesn't bode well for the game industry. Metacritic is famous for scores that a lot of times don't represent the quality of the game they measure. I definitely don't want to be measured as a developer by this system. I've seen companies get caught up in designing to metacritic scores instead of their fan base, I wouldn't want their hiring practice to go the same way.
As is stands, both GameFaq and Moby Games are inadequate databases of credits for developers. If we are to be judged, it should at least be with complete and verifiable data.
In my opinion, in some ways it really feels like Metacritic isn't something that casual developers should fear, but people targeting game developers, or extreme gamers should loathe.
Honestly I had never heard of it till I started hanging out with game developers maybe 4-5 years ago, even now I don't use it, I just look for gameplay trailers.
I've asked dozens of people if they have heard of it, and the only people that had ever used it before were mainly game developers. Students who want to be game developers, or broke guys, who trusted rating sites.
My brother, and sister who buy games for themselves, and their kids don't have any idea what it is. My best friend who despises the internet, but plays games all the time never heard of it, and buys a bunch of games sometimes still just based on the coverart of the packaging. (which I think is crazy, but awesomely simplistic of him)
I've asked kids, and teens passing by in gamestops, and other venues how they choose their games. Most of the time it's word of mouth, or magazines.
Perhaps it is a different story elsewhere, but here in the midwest, few people know metacritic, and yes people still buy games.
Guys with 25 years in the industry and hundreds of titles under their belt are ending up on here with 1 game developed and scores of like 20%... if they're on there at all. Mobygames has been collecting credits info for like 10 years and is still nowhere near complete or accurate. I expect the GameFAQs/Metacritic credits DB to have similar levels of success when it comes to completeness and accuracy 10 years from now.
Doesn't matter, anyone daft enough to use this stuff for hiring isn't worth working for (or hiring). It's a cute, fun, meaningless, and inaccurate statistic.
Perhaps. But at least it's a step in the right direction: recognition of individual creators.
They have the best chance of forwarding the needs of creative. Otherwise, we will remain where we are now: the most powerful individuals in this industry being suits.
I've been in the industry for 31 years, worked on hundreds of titles, and they have a grand total of one score from a game I consulted on for a single week. That's one week in about 1618. Not exactly stunning accuracy.
The ratings for games are never broken down into categories such as "Programming Efficiency", "Artistic Proficiency" or "Pipeline Efficiency" or "Developer Creativity vs. Publisher Demands", so how the hell do you legitimately acquire public data to create performance statistics for specific individuals supposedly involved in the creation of a game in order to somehow "rank" them against other individual developers? I've played plenty of ugly looking games that had fantastic game and level design, likewise plenty of beautifully polished looking games that were terrible to play, so the performance of a final product is impossible to use as any kind of valid metric for guaging the contributions of any one individual involved in a game's development. I'm sorry if I'm preaching to the choir on this, but this whole idea is just plain ludicrous and insulting to all professional and independent game developers alike.
I'd much rather people judge my work from my portfolio as opposed to some arbitrary number, so it's probably a good thing I'm not listed on there. Not that I expect potential employers would be silly enough to that, or if they are I don't think I'd be too confident working for them...
But the real problem I have with it is this: how can they assign a score, based on average scores of the games I've worked on, to judge my 'score' as an environment artist? What has the overall score of a game, let down or bumped up by any number of things outside of my control got to do with my work? *Maybe* if it were an aggregate of graphics scores it might have some slight merit, but even that's a stretch.
I guess what I'm saying is: what do they think they are actually rating??
We were tossing around this really cool idea at GDC: Take classic works of literature, paintings, sculptures, architecture and review them meta-critic style so everyone can see if they want to buy or view them in a museum based on a numerical score. (Note the tongue-in-cheek sarcasm here).
Van Gogh's Starry Night: 67% This piece has good colors but it looks like something my kindergartener made finger painting today in class.
Pisano's Leaning Tower of Pisa: 43% Ok, first off did Pisano really make this thing? I mean someone else is taking credit for it. Get an IP lawyer and sue him so we can be sure our facts are straight. Now about the tower: this is a pretty shoddy piece of architecture. I mean it doesn't even stand up straight. I would not advise buying a plane ticket to Tuscany Italy just to see this. Let's hope the Tower of Pisa 2 gets rid of the "gangsta lean" and works as a real free standing structure. I mean Stonehenge was made sturdier and it was built by like, cavemen or something.
On a serious note, has anyone noticed that on MetaCritic any movie score above 60 is in the "green" zone, yet for games to get into the "green" they have to top 75? Isn't that a little odd, since MC is basically acknowledging a double standard?
You can play a pretty fun game on MC by taking a movie score and seeing what its counterpart is in the game review section, and vice versa. Source Code would be Homefront, Rango would Hard Corps: Uprising, and Blue Valentine would be Blur.
Hmm. Always knew this "Richard Garriott" fellow was overrated. Only a 66? Sure, SOME magazines will claim that that is "above average" but now I guess we all know he'll never make it in this business
This is silly - just another product of a brave new information world obsessed with metrics. It reminds me of the public "ranking" of teachers in California and what a stink that created. Teachers didn't like it either and for good reason. It's a misrepresentation.
Though I guess the exact problem isn't that flawed ranking systems like this exist, the problem happens when people start assigning any significance to them.
It's the quality. And the central importance.
So quality matters far more.
Also, the corrections process hardly ever works. Maybe if they fixed that, they wouldn't have to be so pedantic about credits scans and screenshots.
Were you a programmer on a game with great tech, that flopped at market because of improper release schedule management or a screw-up in marketing? TOO BAD! You're still a 56!
Hmmm.
Though it is sad that this is probably going to incite some of the stupidest fanboy wars ever.
Really? Really!? This is laughable.
As is stands, both GameFaq and Moby Games are inadequate databases of credits for developers. If we are to be judged, it should at least be with complete and verifiable data.
Honestly I had never heard of it till I started hanging out with game developers maybe 4-5 years ago, even now I don't use it, I just look for gameplay trailers.
I've asked dozens of people if they have heard of it, and the only people that had ever used it before were mainly game developers. Students who want to be game developers, or broke guys, who trusted rating sites.
My brother, and sister who buy games for themselves, and their kids don't have any idea what it is. My best friend who despises the internet, but plays games all the time never heard of it, and buys a bunch of games sometimes still just based on the coverart of the packaging. (which I think is crazy, but awesomely simplistic of him)
I've asked kids, and teens passing by in gamestops, and other venues how they choose their games. Most of the time it's word of mouth, or magazines.
Perhaps it is a different story elsewhere, but here in the midwest, few people know metacritic, and yes people still buy games.
Doesn't matter, anyone daft enough to use this stuff for hiring isn't worth working for (or hiring). It's a cute, fun, meaningless, and inaccurate statistic.
They have the best chance of forwarding the needs of creative. Otherwise, we will remain where we are now: the most powerful individuals in this industry being suits.
Here's what everyone should unanimously be saying...
"This is fucked up."
But the real problem I have with it is this: how can they assign a score, based on average scores of the games I've worked on, to judge my 'score' as an environment artist? What has the overall score of a game, let down or bumped up by any number of things outside of my control got to do with my work? *Maybe* if it were an aggregate of graphics scores it might have some slight merit, but even that's a stretch.
I guess what I'm saying is: what do they think they are actually rating??
Van Gogh's Starry Night: 67% This piece has good colors but it looks like something my kindergartener made finger painting today in class.
Pisano's Leaning Tower of Pisa: 43% Ok, first off did Pisano really make this thing? I mean someone else is taking credit for it. Get an IP lawyer and sue him so we can be sure our facts are straight. Now about the tower: this is a pretty shoddy piece of architecture. I mean it doesn't even stand up straight. I would not advise buying a plane ticket to Tuscany Italy just to see this. Let's hope the Tower of Pisa 2 gets rid of the "gangsta lean" and works as a real free standing structure. I mean Stonehenge was made sturdier and it was built by like, cavemen or something.
On a serious note, has anyone noticed that on MetaCritic any movie score above 60 is in the "green" zone, yet for games to get into the "green" they have to top 75? Isn't that a little odd, since MC is basically acknowledging a double standard?
You can play a pretty fun game on MC by taking a movie score and seeing what its counterpart is in the game review section, and vice versa. Source Code would be Homefront, Rango would Hard Corps: Uprising, and Blue Valentine would be Blur.
http://www.metacritic.com/person/richard-garriott
This is silly - just another product of a brave new information world obsessed with metrics. It reminds me of the public "ranking" of teachers in California and what a stink that created. Teachers didn't like it either and for good reason. It's a misrepresentation.
Though I guess the exact problem isn't that flawed ranking systems like this exist, the problem happens when people start assigning any significance to them.
Thats just wrong...
I don't have the resume that many of you do, but these sites are pretty useless if that's the best they can do.