What Went Wrong
1. We Misjudged Our Audience
Prior to SpaceChem, I had already established a small but strong fan-base with my previous "engineering" games. On Kongregate, Codex had over 400,000 plays; it didn't seem like a stretch to convert enough of them into SpaceChem customers to cover our costs of development and make a little money, given that the game launched at $20.
So, when we started thinking about who our audience was for the game and what would be appropriate, we assumed the standard Zachtronics audience, who seemingly wanted a longer, more polished "engineering" puzzle game. It should be noted that this is the same audience that enjoyed KOHCTPYKTOP, a Soviet-themed puzzle game about integrated circuit design and layout.
Fast forward to March 2011: three months after release, SpaceChem was available on Steam to a much larger audience and we were looking at the possibility of making far more money that we had ever imagined, if only we could convince the general public that they wanted to play a game that very much appeared to be about chemistry.
The first incorrect assumption we made was thinking that everyone likes science. Although the internet may love "Science!" thanks to games like Portal, games that look like actual chemistry remind most people of chemistry class.
The number of times I've read and heard the words "but I'm not good at chemistry" in connection with SpaceChem is staggering. A particular game design colleague has asserted many times that had we called the game "SpaceGems" and made it about alchemy, we would have sold twice as many copies. Although I like building games around real-life knowledge, I'm not sure if I disagree with his assessment.
The second incorrect assumption we made was thinking that puzzle games need to be difficult and long to be good. Although challenge is essential to making puzzle games "work", the difficulty that emerges from the mechanics of SpaceChem can be bewildering, even if it's what makes the puzzles feel so open-ended. Combining this intrinsic difficulty with an abundance of puzzles that must be beaten to "complete" the game gives SpaceChem an oppressive 40+ hour difficulty curve that only 2 percent of players reach the end of.
2. We Flubbed Our Metrics
In our first private beta playtest, our metrics capturing system failed to capture any data, an event that in hindsight describes much of our statistics-gathering strategy for SpaceChem. Preoccupied with fixing post-launch bugs, it took us weeks to realize that our demo couldn't upload metrics data.
Later we discovered that, because we only uploaded metrics at the start of the game, we never captured any information from single-launch customers, which likely included many players who got stuck in a tutorial and then never bothered to try the game again. By the time we got things straightened out, we only managed to learn the magnitude of obvious things, such as the fact that SpaceChem is stupidly difficult.
On the plus side, I got to make some neat infographics!
Completion rates for the puzzles in the SpaceChem. (Click for larger image)
The "average" of 500 solutions for a particular puzzle in SpaceChem.
3. We Made the Game Too Long
When making a game, it's important to provide as much content as possible -- right?
The biggest difference between SpaceChem and my previous games is that it was a commercial endeavor. The second biggest difference was the amount of content -- 50 puzzles, plus a full story, instead of the five to 15 story-free puzzles I would usually aim for. Although some of the earlier puzzles in the game can be solved in minutes, many of the puzzles take hours. Some, such as the game's final boss battle, routinely take players days to solve -- if they even get that far.
There's nothing wrong with having difficult puzzles in a puzzle game; the capabilities, interest, and patience of your players will always span a huge range, so difficult puzzles can keep the best players challenged and give everyone else something to aspire to. However, when progression through a story is blocked by progression through the gameplay, making the game too difficult denies all but your best players completion of the story and the satisfaction and enjoyment that goes with it. By the time we realized this, it was too late. Since our story and our puzzles were tightly coupled, separating them so that the story ended earlier would have required reworking the entire campaign.
The puzzles available in SpaceChem fall into two categories: campaign puzzles and user-created "ResearchNet" puzzles. Although I've heard many players describe their lack of progress in the campaign as "not finishing the game", I've never heard anyone claim that, by not beating every single ResearchNet puzzle, they felt as if they had unfinished business. This seems to indicate that players naturally tag the end of the story as the "completion" point. So, if you're going to make a story-driven puzzle game that gets progressively harder, you really ought to put the hardest content after the end of the story!
We ran into a similar dilemma when deciding where to end the demo. Since SpaceChem is so unlike other games, it seemed fair to make sure that players got a good taste of the game before being forced to make the purchase. The puzzles in SpaceChem are divided into planets (about six puzzles each), with the first two planets consisting entirely of tutorial puzzles.
Ending the demo on the third planet had the benefit of giving players a good feel for how difficult the non-tutorial puzzles were and demonstrated the game's boss battles, the first of which did not show up until the end of the third planet. Unfortunately, it also made the demo about four hours long, which isn't good for encouraging impulse buys.
I've heard claims from various sources (particularly in the mobile space) that making your demo shorter is always better for sales. Even if that's true, I don't entirely regret our decision. Any player who realized the game was not for them when they reached the third planet would have been strongly disappointed if they had to buy the game to learn this. Likewise, any player who made it past the third planet and wanted more would be likely to love the rest of the game. Our business exists to make our customers happy -- why would we act against that?