 |
|
|
| |
Opinion: Are Starcraft II's Ladders Its Biggest Flaw?
by Chris Breault [PC, Console/PC, Exclusive]
|
|
| |
|
April 13, 2010
|
| |
[In this opinion piece, Chris Breault examines his experiences playing Blizzard's recent Starcraft II beta, suggesting that it's the key ladder-based system behind the much-awaited PC RTS which'll mean the difference between joy and frustration for its legion of players.]
Starcraft II is likely the most ambitious feat of RTS engineering ever attempted. Blizzard aims to strike a near-impossible balance between the desires of the hardcore, who have played Brood War for over a decade, and the new, larger audience they plan to attract.
But for all the care and craft put into the game, Blizzard hasn’t been able to manage this tension, and the beta shows this. Few people outside of the SCII beta forums have discussed it, but a big piece of the game’s multiplayer is seriously dysfunctional.
An elaborate ladder system governs SCII matchmaking. The ladder tells you what league you’re in, your division within that league, your current rank within that division (out of around 100 random players), your point total, and your record of wins and losses.
You can’t separate the ladder from the game. You see your league, rank, and win record every time you visit the “Multiplayer” screen. The ladder is your character status screen; like WoW, SCII gives you “rested experience” that will be added to your point total when you win after time away.
The ladder is the hook that keeps you playing all night, grinding short 1v1 matches, gathering points, and clawing your way to the top of the rankings. You can't appreciate the seductiveness of this system until you've put in the time to get near the top of your league, which few game critics seem to have an interest in. (Only Olly Quinn details his ongoing project of attaining "competence.") But the more climbing you do, the more wobbly the ladder looks.
After placement matches, you’re dropped into one of five leagues and ranked according to your performance in your division. You want to ascend to the first rank in your division, from which point you may move on to a higher, more skilled league. In theory, at least.
In practice, you play everybody else from every division, and the system rarely matches you with someone from your own. You actually feel like giving your division-mates a high-five when you’re matched with them, like two people from the same town who run into each other in the big city. Blue posters on battle.net forums have defended the system, saying it makes people feel better about themselves ("I'm 4th in my division!").
But only credulous players will find satisfaction in this, because you do not even compete directly with the people in your division. You compete with everyone, and the system compares you to a tiny, random sliver of that population. It seems to calculate your point total relative to your division, not your league, even as it matches you with people around your league, not your division.
Say you ran a race, along with 20,000 other people. You finish behind about 7,000 of them. You look at the standings and they say you finished 4th. In a move to boost participants' self-esteem, the race's organizers have divided the standings into 200 randomly selected groups. Would you really feel better?
When Blizzard designed their ladder, they had a choice. Would they create an accurate leaderboard and risk alienating a new player, who would see the thousands of Starcraft players so much better than himself? Or make a ladder that deliberately concealed information, and didn’t risk damaging the ego of a fragile noob?
They went with the latter, more condescending option: the sole purpose of battle.net divisions is to hide data. It’s a hollow move for a company that prides itself on commitment to “e-sports.” They preserve the form of a ranking system, while ignoring its function: telling you whether one player is actually better than another. This system, meant to bridge the needs of the hardcore and the casual, will satisfy neither.
As you climb upward in your division, competitors from higher leagues appear. I've beaten Bronze, Silver, and even a few Gold players, yet I was never promoted from my position as #1 Copper. The loading screen tells me that I am "Slightly Favored" against most Bronze players it pairs me with; if I see "Teams Even," I may face a high rank Bronze or a middling Silver. In other words, the game's odds-maker routinely tells me I am on even footing with people who are literally out of my league. They are, in some cases, two leagues above me.
Many cases are much worse. Consider szcz, a #1 Silver who routinely plays and beats the top Platinum players, and is even “Favored” against some of them. What lunatic devised a ladder that evaluates player performance (to determine odds) with a system different from the one it uses to actually rank them?
The rewards system does not seem to rely only on wins and losses. A Blizzard employee, Bashiok, posted the following FAQ:
Q. How does a player move from one league to another?
A. After you’ve finished your initial placement, the system continues to review your performance and determines what league you should be placed in based on those reviews. The time and frequency of these reviews is kept hidden.
As is all description of the data under "review." Forum posters hypothesize that the scoring might be based on actions per minute, unspent resources, and other trivia recorded by the game, and speculate that Blizzard keeps the conditions secret so players can’t exploit the system. But a system based only on wins and losses wouldn’t be exploitable (except by literally cheating), and at least then players would know their goal should be “winning games.” Under the current system, players are told to meet a secret set of conditions to advance. It’s not a fun game to play.
Here’s my beta story. When I started – 200 games ago, playing worse than I do today – I was placed into Bronze League. I played some games, lost several, but won enough to stay well above the bottom of my division. Then I came out of a winning game and received a message saying that I had been bumped down a league. I was demoted from the victory screen, and no reason was given.
When the game should have congratulated me, it kicked me in the balls. Blizzard developers have a reputation for creating powerful reward structures, but the SCII team hasn’t shown that talent; their rankings and rewards are as arbitrary and obscure as possible. In WoW, the remaining work needed to advance a level appears constantly on the default UI. It’s a goal the player understands and will not forget. In SCII, rewards are an intentional mystery; like miracles, you can only hope they will happen.
Better concepts aren't hard to work out. Make a global ladder for each league. Do not match people from different leagues together. When the top ranked players in the league start dominating mid-ranked players, mark those top players for review. Put them through placement matches again (maybe fewer this time) and let them place into another league.
The game isn’t finished, and Starcraft wasn’t built in a day, after all. But nobody from Blizzard has acknowledged that there is anything wrong with the ladder, or that they are working to make it more transparent. They need to. If there’s one thing both veterans and new players hate, it’s a lack of communication.
[Chris Breault is a gamer and freelance writer. He maintains a blog at http://post-hype.blogspot.com, and can be reached at post.hype@gmail.com.]
|
| |
|
|
In relation to the blog:
I have been a true fan of RTS games and of course, Starcraft. I got the great chance of playing SC2´s beta and I have to agree with this blogger on the ladder system. You just don´t know why you got promoted or demoted on your bracket. I got the chance to be 1st or 2nd on bronze league 2v2 and wait 3 or 4 days to get promoted, and get instantly promoted on 1v1 when I was 4th. I havent been demoted yet, but after reading this post I got to think about the situation when it would happen and it wouldn´t be nice at all if I didn´t know the reasons behind it.
The way this ladder works takes me back to the times when WoW´s Battlegrounds weren´t cross server and you had to fight against opponents from yours. Sure, it took longer for games to come up, but they had extra spice when you met the same opponents and had the chance to create a relationship good or bad, with them (the case fighting against people from your league in SC2). When they cross servered Battlegrounds, games queued way faster but that spice was lost. I have to assume the ladder design is focused towards efficiency and not to any sense of cohesion between players of each league.
Ive liked RTS games but I dont consider myself a hardcore player, and I have liked SC2 so far. I get the feeling hardcore players will like it as well, since this ladder system, even though confusing, works for the ones that have the skill and spend enough time with it.
Well, you're already wrong there, because I've been completed addicted to it -- even when I was clearly placed in a ranking above my actual skill level and had to fight to keep it. When I play good, I have moved up. When I play bad, I move down. The ladder is not that much of a mystery unless you begin to obsess over its finer details. Which I find usually occurs after losing 20 matches in a row, or, say, getting knocked down from Bronze to Copper.
As far as the split divisions, I'm thankful for that. I'm glad that data is hidden, because yeah, it would be pretty depressing to see my rank in the thousands. I like competing against a smaller subset within a larger league. In the end, what difference does it make? I know I'm not going to be in the top 1,000 players in the world, so give me something smaller to strive for. It may be clever Blizzard trickery, but it's also what I want.
Note that the truly hardcore players will have an invite-only division to play in. It's also probably important to note that the game is still in beta, and they have clearly been tweaking matchmaking to a considerable degree. For instance, the experience you mention likely came during an early wave of the beta, a phase that was run pretty differently to what is currently being simulated.
I can't agree with Stephen Dinehart regarding the game's quality -- I think it's a brilliant refinement of one of the best video games ever devised -- but I do agree with his assessment that matchmaking and Battle.net at large are the best features of the game.
You're in the minority in your desires, but luckily there seem to be other developers aiming to meet them.
I don't think Blizzard is giving people what they want with this game. It has only one thing that I want and that is a new SC game. Other than that it has no LAN game play and the story mode was reduced to only one race per release as far as I understand. That is not what I want nor is it what any of my friends want. We many times get together to play games on LAN and usually we go to a place that has no internet so having SCII is worthless during these trips if it does not allow LAN games. Even if we did have Internet access, having 8 computers (Outside the US) using the same internet connection that has to travel all the way to the US and back to transfer data between our computers would generate a Lag that might make the game play impossible.
People like rankings (I have nothing against them, But, I'm not one of them) but I prefer the option to choose ranking globally or non ranking free friendly play in LAN.
They are trying to bridge the gap between hardcore players and more casual or normal players. The division system is a great way to fix the ranking for the "normal" players.
However, it ALSO needs a way for the hardcore to rank themselves on a global ladder. Hardcore players want that; casual or normal players like myself, who do not expect to get better than a five- or six-digit global rank, do not.
Re: Your more specific points:
Being bumped down after winning a game: This is a clear bug that they should address. They could simply make it wait until you lose before bumping you down.
As for not matching people from different leagues: That's something you have to do to determine if players should move up or down leagues (e.g. if you lose to bronze when you're silver, you might move down to bronze). But also, when it launches, there will be way more players so you should get matched more evenly and consistently against players at your level.
The idea of redoing the placement matches is overly complex. They can already evaluate you based on your normal matches, and it would just add complexity (though I admit it might be fun to play some matches where you know a league upgrade's on the line).
The casual players, they don't actually care what their rank is (well if you don't paste it up on the opening screen they have to stare at everytime they log in). What casual players want is a 'good game' and that means a game that is relatively fair with opponents of similar skill and style so it becomes an epic contest of blows back and forth culminating in a unanimous 'GG'.
Casual gamers want a match making service that matches them in lag free environment with players of the same skill, they don't want to fight the totality of the system, they want to be syndicated off with like minds so when they match they have balanced experience as opposed to a random systematic curb stomping from the platinum crew to remind everyone of their place in the pecking order. That's just a waste of everyone's time and teaches nothing other than an exercise in grief.
I might be wrong. I don't play RTS games. My belief that Blizzard will create what most current and potential customers will want to buy and be satisfied with is based on their history of actions which generally supports this. However, I agree that the lack of LAN play is absurd and I am surprised to hear about it. Could they be straying from the principles that served them so well for so long?
Chris--I'm conflicted about the ladder. Your point is very well-put. I think the idea of one big ladder is a tough one; I feel more invested in my progress with the current system because it feels like I have a legitimate chance to actually work towards something, whereas with a single combined ladder (even stratified by league) I can almost guarantee based on past RTS experience that I won't move much, especially as the higher-level players solidify their positions. But you're completely right that, in reality, the current system isn't really concretely indicative of any particular ranking at all, and is essentially arbitrary. I'll be interviewing some Blizzard folks soon, and will be sure to raise this issue.
I'm confident that the match making system will match you to more even opponents when your user base is not 10,000 and is over a million. If you look at the total players online at any given time, you will see about 10,000 people in 3-4 thousand games. Given that its a 2 or 4 player game, that's not a very big pool of available people to choose from in the looking for group channel...
Given that Starcraft is likely a more complicated game than Chess with as much of a variance in player skill level, statistically with this few available player to choose from, it just makes sense its functioning the way it is!
I love what they are trying to do with SCII leader boards. Will it ship with problems, likely. Will Blizzard get it right within the first 6 months, very likely!
I'm surprised Blizzard, of all the game companies, succumbed to the whole 'hold the newbs hand and don't let them feel bad about themselves' silly mantra at the sake of a logical implementation of a gameplay system. I sort of thought they were above that... I hope this doesn't bode well to some of the decisions behind Diablo III , which is just about the game I'm most looking forward to this decade.
This division system sounds pretty dumb to me, I'm surprised they tried it out. I'll be even more surprised if they stick with it. Sounds like if they stick with it they are just being obstinate about it, because any outsider can see that it does not work.
Here's my free solution: have a game ranking system out of 100. Each 'rank' is actually a level, with X many of players on it (you can keep this hidden). Each newb starts at rank 100. Works way up to level 1, with their secret formula. The rank ladder is pyramid shape. So newbs don't feel bad about being rank 100, and the ladder actually means something, and you can keep the specifics secret in order for balancing and rearrangement. Each country has its own ladder.
You can choose the difficulty of your next opponent, from a scale of -5 to +5 , below or above your current ladder rank. Your ladder-score is from the match is tallied accordingly with this choice. So if you challenge some +4 levels above you , you get a 4x multiplier to your score. If you challenge some one 5 levels below you, you get 20% of the ladder score as if you challenged someone on your same level.
Finally any title has to attract new users to the base. Period. This means there has to be some acceptable method of participation and integration into the community. Focus solely on core membership hits a company in the one place you can't ignore and that is the pocket book. That's the reality you have to face and makes new IP franchises both risky but allow the most freedom.
On my analysis, it's Blizzard--they're doing far more right than wrong at this point. If the arc of their title development catalog progresses along with their battle.net plans, you're looking at a solid set of titles that will probably set the parameters of much of the on-line gaming industry for the next 2-5 years for their particular business model. (Though I do some give in the F2P and social area if they do choose to grow in that direction. I suspect it will be consoles NEXT year based upon this years title release performance which will leverage the Activision Blizzard partnerships if they aren't already being leveraged.)
Starcraft 1 is quite beloved for me, and I will still defend that sprite-based 12-year old game as "the best RTS thingy ever". I was so excited for Starcraft II, when I didn't get a beta key I dug into the groups online who were cracking how to play it solo vs. AI. There were a lot of things I liked and a few things I didn't. But overall I felt like it was true to the original and I was really looking forward to the campaign.
Then I got my beta key.
I log in to this ridiculous ladder system which is only somewhat understandable AFTER my brother explains it to me at length; I enter the practice league hoping for some easy matches to get me going, only to get "unable to find a partner" and then wait a while before getting matched with people who obviously know these maps like the back of their hand and have been working with newer strategies I hadn't even heard of. The one time I get someone who's probably never even played any SC before he was my teammate.
And so all my burning desire to play SC2 has been drained, at least for the multiplayer. I have no desire to participate in this ladder system, I have no desire to play competitively at all! I love SC and am decent at it but I'm not looking to be a pro at the game. More and more games these days seem to want to shove a W/L ratio in my face and compare me to everyone else, when I just want to be able to login now and again and play a quick fun match.
It feels like the more "sophisticated" matchmaking in games get the more broken they are. Just give me a server list and let me jump around. I'm going to go play TF2 now...
The ladder is fun in practice, and should really hit its stride (especially for n00bs like the author) once it launches and has millions of people playing.
I can see how those placement matches can be frustrating for n00bs, but once you take your licks it will start to match you against people of similar skill.
Face it: Starcraft 2 is going to be the golf of our generation. And zomg you can play it over a LAN if that LAN can connect to the internet, and the only lag you might see is in the matchmaking/lobby, not the game itself since it doesn't rely on blizz servers during play.
The true mark of an excellent matchmaking system as suited to my purposes is one that puts me up against people who perform roughly as well as I do. The second most important thing is that it does this quickly. The third most important thing is removal of abusive and cheating players.
What keeps me interested in the game is its ever self-adjusting difficulty that feeds from this system. It's not too hard nor is it too easy. I turn on the game, I hit play, and in a matter of seconds I'm having a good time. You'd be hard pressed to say that about a lot of multiplayer games these days.
I can't believe Blizzard is still even releasing patches for Diablo II. Very commendable of them.
And all without a dime changing hands from the consumer to developer. (Unless there is a black market for public beta keys that I am unaware of...)
I know I keep harping on this across several posts but this is the Apogee model of play THEN pay. This is important because I do NOT see this model reproduced anywhere else, even free to play models where the monetization is different, but still produces sizable revenue.
So I guess what I am saying is that the Ladder system, while not ideal, is certainly something there has been the least amount of feedback on apart from unit and map level balancing. If you like this delivery method of beta to market, then this had better work out and be profitable.
In short, I disagree with most of what the author of the original article / blog is saying. From reading through numerous threads, to me it is apparent Blizzard's system will well serve the bulk of its customer's in practice. Let's consider firstly what the goals are of a good rating system. From this standpoint, we can then evaluate Blizzard's design choices for Battle.net 2.0.
First, any good rating system will stratify players according to their approximate skill levels. Thus players will have a good chance (when selecting auto-match) of being paired against opponents of comparable skill. This gives players a challenging but not psychologically devastating experience: when the ratings settle over a large population of players, the vast majority of people will win 40-60% of their games. Why is this? Because a good rating system is based on probability. The computer therefore knows when it matches you up (assuming your rating is a good measure of your skill and likewise for your opponent) the most likely outcome of the game. It can therefore deliberately select opponents in your skill range; i.e., opponents who you will have a 25-75% chance of defeating.
Second, a good rating system will be able to evaluate a player's skill based on his or her performance with a relatively small number of games. It is unrealistic to expect this to be possible in just 10 multi-player games, though! If you haven't played 50 games, your rating in any system will be highly uncertain. This uncertainty will decrease with time as you play more games, allowing your rating to more precisely measure your skill level.
Third, a good rating system will allow rapid advancement of players who are demonstrating real improvement in their playing ability. This means allowing players who are winning games against higher rated players to be able to earn points rapidly, and thus reach a rating more representative of skill level. It is often the case in skill-based games that improvements don't occur linearly but in bursts (i.e., a step function). At some point, the brain clicks with a new way of thinking about the game, and improvement in performance can thus occur rapidly. A good rating system can track this effectively.
To counter-balance my third point, however, a good rating system is stable. It thereby provides a real benchmark of progress in learning the game. You know when your rating increases significantly that it is an indication of improved skill and not just luck because to advance up the ladder significantly, you have to demonstrate consistently the ability to compete with better players.
Now to the examples of Warcraft 3 and chess before returning to the original point of why I think the Battle.Net 2.0 system will work fine. First, I am an "A" player in chess with a rating from the US Chess Federation of 1837, which puts me in about the top 15% of rated US players. So, I'm nothing special but am somewhat above average.
The categories above me are "Expert" (2000 - 2199), "Master (2200 - 2399), and "Senior Master" (2400+). The best players in the US, nearly all pros (or were at one time), are in the 2600-2800 range. It is very rare for an amateur player to exceed 2400. Incidentally, 2200 strength is almost exactly the 99% point in the ratings distribution. Even though the exact rating precisely describes a player's performance to date, people in the chess world very frequently use the terms above to describe players' skill levels: E, D, C, B, A, Expert, Master, and Senior Master.
To continue the chess example, my rating has been stable for a long time but has recently started to increase (fairly rapidly). Why? Because I picked up a few chess books from Amazon.com and started reading them. I am not drastically better than I was, but my performances are a little improved. But my win ratio is very consistent---a sign of an excellent rating system.
In season 3 of Warcraft III, I reached my peak of level 25 on the solo (1v1) ladder. At that time, I was playing people well matched to my playing strength and winning about 50% of my games. I was "stuck" at WC3 Level 25 because I had not made the mental leap required to significantly improve my skill. I did not just get rewarded by rating increases for playing hundreds of games. To increase above 25, I would have had to show real progress in playing ability, which I never achieved. Hence, the ladder worked perfectly for my case.
So, let's look at the Starcraft 2 system by comparison. Blizzard is assigning people true skill ratings. But they're also assigning people to "leagues" of medals. This is pretty much the same as A-player, Expert, Master, etc., in chess. It's a courser way of describing someone's skill that has independent value to the rating itself. Why? In chess, the reason is clear. In the chess rating system, 200 points is the threshold of what one would consider a significant difference in playing ability. Mathematically, a player rated 200 points above me has a 75% chance of winning the game.
So, it seems to me the Starcraft 2 system presented in the Beta encapsulates the best features of the rating systems I've seen. People are consistently saying that after a significant number of games (25+), they start to get paired against players of comparable strength. Furthermore, as in chess, what league you're in is a rough estimate of your skill, though of course your rating backs it up.
Finally, I have read in Blizzard FAQs that the division rankings have nothing to do with who you play. They're just a way to let you know how close you are to the next league above you. The author also complains about not getting paired with people in his league. But why should the match-maker be restricted to this? How else would mobility through the ladder be possible? Leagues are just a rough indicator of your playing strength, just like the term "Expert" or "B-Class" in chess, nothing more.
With all this (admittedly overlong post), I wholeheartedly defend Blizzard's decisions and fully expect to get a great MP experience from SC2, if the game itself is any good.
It shouldn't be possible for you to rank up in your division (or stay stable), and down in the league, not unless you are towards the bottom of your division. The concept will befuddle players, and seems like an over complication of the system. It certainly shouldn't do it to you just after you've won a match!