[This article by Ryan Henson Creighton is re-posted from the Untold Entertainment blog, which is awesome.]
Have you ever run across a video game or movie that was wildly mis-marketed? Many players expressed their frustration after playing recent indie game releases Dear Esther and Proteus because they weren't gamey enough, and countless moviegoers have been lured into theatres to see Kevin James or Adam Sandler movies that the trailers would have them believe are actually funny.
Don't buy into false advertising: Every Adam Sandler "comedy" is actually a tragedy.
While testing Spellirium, our upcoming point n' click graphic adventure / word puzzle mash-up, i started to make many of the same mistakes i made with past games: relying too much on the advice of my game dev friends who weren't interested in the genre to begin with, and telling myself that the game just needs to find its audience to be appreciated. i'm determined to correct those mistakes with Spellirium. This is the story of how i plan to do it.
List Your Turn-Ons
i faced many challenges testing Interrupting Cow Trivia a few years back, and while i learned a few important lessons, a number of things remain a mystery to me. The most important thing that ICT testing taught me was to weigh testers' feedback according to how "into" the game they are. If you asked a casual puzzle game fan like me to playtest Gears of War, you wouldn't necessarily get the kind of feedback to make a better Gears of War game ... you'd only end up making an unsuitable game slightly more palatable to a casual puzzle audience.
NOW we're talking!
i revised my feedback survey for ICT testers to begin with the question "Do you like trivia games?" If the tester answered "no", the rest of his feedback would get shuffled to the bottom of the stack.
A 5-Letter Word for DERP
i've been testing Spellirium with people who aren't word game fans. How do i know? There are a number of "tells". The most obvious is when it takes a player forever to build a word. Spellirium gives you a 49-letter grid, and you can make words from 3-8 letters in length using any of those 49 letters, in any order. When a player struggles to make a 3-letter word, i know something's up.
(i can make a couple of 3-letter words from that first row alone)
If the player has no trouble making words, there's another "tell" that outs the player as somewhat of a non-wordgamer: the player makes a long 6- or 7-letter word using "common" letters, and is disappointed he's not supremely rewarded with Peggle-style fireworks. i've had a few testers complain (or express surprise) that a word like "TESTERS" scores lower than a word like "POX". Of course, any Scrabble player will tell you that it's more rare/unique/difficult to use high-value letters like P and X in a word, than with common final-round Wheel of Fortune letters like RSTLNE.
The issue of players' reactions to high-value letters was apparent with two iOS word games that were released around the same time last year: Puzzlejuice and Spelltower. Puzzlejuice creator Asher Vollmer told me he actually bowed to player pressure and changed the game's scoring mechanism to reward longer words instead of words containing high-value letters. Spelltower, meanwhile, becomes more difficult as the grid fills up with X's, Z's, Q's and K's, implicitly reinforcing the idea that these letters are tougher to squeeze into a word.
Puzzlejuice and Spelltower: two different approaches to the letter value problem.
So through Spellirium playtesting, i kept telling myself that i just needed to get the game in front of the "right" type of player - that those who would like it, would like it a lot. Unfortunately, that's not at all how the market works.
To Market, To Market, to Buy a Fat Game
The way the market actually works is that you catch wind of a game through a friend or a website, and you eventually stumble upon its page on a digital distribution site like Steam or Good Old Games. You watch the trailer, look at the screenshots, maybe double-check its purported quality by reading Metacritic reviews (or just glancing at the game's damnable Metacritic score) ... and you imagine what the game might be like to play, and whether you'll enjoy it. You create a mental picture of that enjoyment you'll get from the game, and then you compare that to the asking price. If the asking price is aligned with the enjoyment you predict you'll get from the game (and everyone's equation for this is different), AND you have that money to fart away on entertainment, THEN you may just complete the purchase.
So if that's how game sales actually work, it makes more sense to me to simulate that environment, gauge potential customers' value equations, and then determine from their testing feedback whether the game delivered on their expectations. So the approach i'm taking now is to mock up the sales page for Spellirium as if it were currently for sale on Steam (to be absolutely clear: it isn't. Yet.).
i'm going to show potential testers this page, and then ask them a few questions:
- What's your level of interest in this game?
- Which aspect(s) or features of the game interest you the most? The least?
- How much do you think this game costs / what would you pay for this game?
- List another game that is like this game. Tick this box if you've played it. Tick this box if you've enjoyed it.
i may A|B test this with an image that shows a price for the game, and one does not. For the potential testers who see the price, i'll ask:
- Would you buy the game at this price when it was released, or would you wait a few months for a sale?
- How many hours of gameplay would you expect to get from this game at that price?
- How do you feel about the price of the game compared to its description, trailer and screenshots? Too low/too high/just right?
- (if respondent answers anything but "just right") How would you price the game?
If i were to approach this exercise completely cynically, i would continue to tweak and refine the page until i got the best potential conversion from my respondents, and then release Spellirium without making any changes to it. Because, speaking absolutely cynically, it doesn't actually matter if the game is good or bad - it only matters that people buy it. But that's not how Untold Entertainment rolls!
i try not to be that guy.
Of course, i desperately do want to make a good game. So i'll use the Steam page mock-up and survey as a funnel to decide on my testers. Those respondents who report the highest interest in playing the game, and the highest likelihood of buying it, will test the game. At that point, it doesn't matter who is a "proper" word gamer and who isn't: what matters is that i have an obligation to the people who are excited about my game and who want to buy it. If those players struggle to make 3-letters words, and if those players expect long words to be rewarded over tricky words, then i will adjust the game for the sake of those players. Because those players are my paying audience - not some mythical "perfect" players that i've hand-picked to enjoy Spellirium the specific way i've configured it. The players choose my game - not the other way around.
It's the sale page and my surrounding marketing efforts that attract the player. i need to make sure that the player i attract is happy with the object of that attraction.