|
About 5 days ago, I noticed something odd. After hanging out around the #6 top rated slot on the Xbox Live Indie Games US dashboard for a month or two, our game's rating started to drop very quickly. In less than a week, our game went from being #6 top rated to our current spot of #11.
I did a little investigating and it turns out that 5 days ago, the "College Lacrosse | The Video Game" Facebook page (which has about 175,000 fans) requested help from its fans to raise the ratings of their games. Now, there's nothing wrong with asking your fans to rate your game positively, but it looks like some of their fans were overzealous and started 1-starring other high rated games in attempt to make it easier for the Lacrosse games to rise through the ranks.
Now for something like Z0MB1Es (the #1 rated game) which has over 50,000 ratings, a sudden influx of 1-star ratings isn't going to do much, but since our game has been out for less than 3 months and has far fewer ratings (about 1500) than the other top rated games , it affected us big time and we dropped 5 ranks. In contrast, the Lacrosse games which were in the middle of top 20 best rated (I don't remember the exact positions) are now ranked #4 & #5.
Note, that ranking isn't just a matter of prestige, it's also a matter of money. The higher rated your game is, the more likely it is that you'll continue to get long-term sales. Our first game, Breath of Death VII, has sold nearly 50,000 copies in the year or so it's been out and the vast majority of those sales (over 75%) occurred after the first 30 days. Before the ratings attack, we had one game that was easily visible in the first screen of top rated #4) and one that was partially visible (#6). Now, we have one that is partially visible and another that you have to scroll over a screen or two to see.
If the Lacrosse developers had explicitly told their fans to attack other games, that'd be one thing, but as far as I can tell, they didn't. They just told their fans to rate their game highly and gave instructions on how non-XBox 360 owners could create a free account and use it to rate the game. Of course that raises another question - should people who don't even own the system be able to rate games on it? With the current system, anyone can rate anything - even if they have never even downloaded the trial. Should this be changed so that only people who have played the game or tried the demo can rate the game?
I have no idea how this sort of thing could be prevented, but I'm really upset. We were really hoping that the upcoming enhanced version patch for Cthulhu Saves the World would give our game the little push it needed to finally stabilize in the top 5 rated. Now, it looks like we'll be lucky just to get our old rank back.
What is to be done about such situations? Starting a rating war with fans of other games 1-starring the Lacrosse games is no good - that'll just make the mess even bigger. What is to be done?
|
Of course not! It's an idiotic design. Several suggestions have been made to change this and the one that makes the most sense to me is to have people that have bought the game (actually bought it, not used a code) weighted normally and people that have downloaded the trial weighted slightly less so. People that have not at least downloaded and played the trial version should not be able to rate at all, period! :
Your stance is quite understandable; if you bought the game, your opinion should matter more. I agree with you there. I'm not sure how you would weigh people differently, perhaps you could assign different colored stars for people to rate with. This could be done when you sign in, the system would know if you own the full game or demo and assign the correct colored star. Say gold for purchased based reviews, silver for demo owner reviews, and a different color for reviews from non-owners.
Why non-owners? Because this is America damn it and anyone should be able to voice their opinion. Suppose I went to my buddy's house and played on his system. I really like the game, but I won't buy it for myself. Suppose also that I want to share my opinion and tell everyone on XBLA that the game was good, but I don't own it. Why shouldn't I be allowed to rate it? Maybe the color of my non-owner rating star is red or pink, maybe I don't like red or pink and it shames me into downloading the demo in order to rate with a silver star.
I'm being silly of course, but the point is that we can always find solutions to our problems. We just have to work together and collaborate, excluding people may not be ideal.
rey ;)
I guess if they did implement some kind of separated review system, non owner reviews should have zero bearing on top rated lists. At least that is what I think.
-How the people who ended up paying money for the game liked it
-How people liked it after playing the demo
-What people who did neither think of the game
That way, no abuse, and more information for consumers. Slightly more work on Microsoft's end, but much better for people using the XBLIG system.
So in your opinion is the Nielson TV ratings system unAmerican because every TV exec can't just buy up a bunch of neilson boxes and plug them into their own TV?
Also, how are you "not sure how you would weigh people differently"? It's called a weighted average.
My god your comment was idiotic.
Is there no fraud reporting option with XBLIG? If there isn't there should be. I think any developer who has lost positions in the top games list due to 1 star reviews since the Lacrosse promotion should be reporting it to Microsoft.
This episode should prove how the system can be gamed, so I hope Microsoft takes some steps to safeguard against this sort of thing. I would go so far as to say that you need to have downloaded the demo in order to rate a game.
A Top Grossing list would be a welcome addition as well, but that's another topic...
Regardless...
Why not fight fire with fire? Use this opportunity to get a gamasutra army together and up your game's rating? And maybe a few of us might even give that lacrosse game a 1star rating. If microsoft isn't going to fix the system, then you have the right to work the same angle this lacrosse game is. For that matter, if there are any games in the top ratings that you don't believe should belong there, go rate it low. But I suppose I'm a little less shameful than Robert. He seems like a pretty humble and reasonable guy.
I'd like to follow up on my comment to say I don't think the Lacrosse guys necessarily had any hand in the downvote bomb. They probably just have some dedicated fans who wanted to see them rise in the list.
If a bear breaks into your tent to steal food, you can't blame the bear, you should have put the food in the storage box. Humans are humans, it's the system that needs changing.
Similar applies to the console dashboard, as it's modelled on the WM7 UI; as a result, information is broken into multiple pages (as with the old dashboard, but at least that offered a "stacked tiles" view which gave you a bit more visibility), each with minimal SDTV-friendly information on it. As such, casual browsing is a slow and frustrating process.
(and from a personal viewpoint, it's also annoying that the marketplace menu defaults to "buy", not "download demo" - and it also doesn't detect if you've already downloaded the demo or not. Which is a step back from the old dashboard...)
However, the icing on the cake is the fact that while full information on a given game is available from the marketplace view, most of this information is unavailable from the "downloaded games" view. Most frustratingly, you can't rate games from the "downloaded" view; instead, you have to navigate into the depths of the marketplace system.
All told, the current WM7 UI generally had a negative impact on the marketplace, both for XBLIG and XBLA games. Roll on a refresh!
Back to the main discussion: fixing vote manipulation. Restricting voting to people who've downloaded the game is an immediate quick-win; it'd stop the problem Ian Fisch mentions, of people creating multiple accounts on the xbox.com website. However, it might not have that much impact on the "downvote bomb" issue, as they're targetting specific games; it's not that hard to download a dozen games, rate them and then delete them.
One solution to this is to restrict voting to people who've bought the game (as opposed to just downloading the demo). However, as mentioned above, this tends to lead to only positive votes being given; few people buy a game because they dislike it! Implementing this approach would therefore be counter-productive, as it'd make the ratings useless.
Instead, what I'd suggest would be to go for a hybrid system:
1) Restrict voting to people who have downloaded the game (demo or full version)
2) Allow unrestricted voting in the first X days after release (e.g. 14 days)
3) Limit the number of votes which can be cast per day after that - either with a fixed level (e.g. 10 votes per day), a sliding scale (50 votes in the first month, 25 votes in the next month, 10 votes after that) or a percentage (e.g. 10% of the number of existing votes)
This would allow the game to get a (hopefully) fair assessment during it's launch period and would then minimise the impact of vote bombing; it's one thing to click a couple of icons on a website, it's another thing altogether to have to download the game and then wait for several days/weeks until you can register a negative vote.
It's not a perfect solution, and some number-crunching would be needed to decide what level the vote-limiter should be set. But I think it'd offer a reasonable balance between being functional and protected...
Vote on this issue here:
https://connect.microsoft.com/site226/feedback/details/619523/game-library-lacks-rating-s ystem-difficult-to-rate-games-after-youve-played-them
I woud also suggest commenting on this issue with this rating abuse story.
It's unreal Microsoft created this problem and has not fixed it. When you reduce the number of proper ratings, you increase the effectiveness of bad ones.