Gamasutra: The Art & Business of Making Gamesspacer
Postmortem: Zachtronics Industries' SpaceChem
View All     RSS
August 22, 2014
arrowPress Releases
August 22, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
Postmortem: Zachtronics Industries' SpaceChem

June 13, 2012 Article Start Previous Page 3 of 4 Next
 

What Went Wrong

1. We Misjudged Our Audience

Prior to SpaceChem, I had already established a small but strong fan-base with my previous "engineering" games. On Kongregate, Codex had over 400,000 plays; it didn't seem like a stretch to convert enough of them into SpaceChem customers to cover our costs of development and make a little money, given that the game launched at $20.

So, when we started thinking about who our audience was for the game and what would be appropriate, we assumed the standard Zachtronics audience, who seemingly wanted a longer, more polished "engineering" puzzle game. It should be noted that this is the same audience that enjoyed KOHCTPYKTOP, a Soviet-themed puzzle game about integrated circuit design and layout.

Fast forward to March 2011: three months after release, SpaceChem was available on Steam to a much larger audience and we were looking at the possibility of making far more money that we had ever imagined, if only we could convince the general public that they wanted to play a game that very much appeared to be about chemistry.

The first incorrect assumption we made was thinking that everyone likes science. Although the internet may love "Science!" thanks to games like Portal, games that look like actual chemistry remind most people of chemistry class.

The number of times I've read and heard the words "but I'm not good at chemistry" in connection with SpaceChem is staggering. A particular game design colleague has asserted many times that had we called the game "SpaceGems" and made it about alchemy, we would have sold twice as many copies. Although I like building games around real-life knowledge, I'm not sure if I disagree with his assessment.

The second incorrect assumption we made was thinking that puzzle games need to be difficult and long to be good. Although challenge is essential to making puzzle games "work", the difficulty that emerges from the mechanics of SpaceChem can be bewildering, even if it's what makes the puzzles feel so open-ended. Combining this intrinsic difficulty with an abundance of puzzles that must be beaten to "complete" the game gives SpaceChem an oppressive 40+ hour difficulty curve that only 2 percent of players reach the end of.

2. We Flubbed Our Metrics

In our first private beta playtest, our metrics capturing system failed to capture any data, an event that in hindsight describes much of our statistics-gathering strategy for SpaceChem. Preoccupied with fixing post-launch bugs, it took us weeks to realize that our demo couldn't upload metrics data.

Later we discovered that, because we only uploaded metrics at the start of the game, we never captured any information from single-launch customers, which likely included many players who got stuck in a tutorial and then never bothered to try the game again. By the time we got things straightened out, we only managed to learn the magnitude of obvious things, such as the fact that SpaceChem is stupidly difficult.

On the plus side, I got to make some neat infographics!


Completion rates for the puzzles in the SpaceChem. (Click for larger image)


The "average" of 500 solutions for a particular puzzle in SpaceChem.

3. We Made the Game Too Long

When making a game, it's important to provide as much content as possible -- right?

The biggest difference between SpaceChem and my previous games is that it was a commercial endeavor. The second biggest difference was the amount of content -- 50 puzzles, plus a full story, instead of the five to 15 story-free puzzles I would usually aim for. Although some of the earlier puzzles in the game can be solved in minutes, many of the puzzles take hours. Some, such as the game's final boss battle, routinely take players days to solve -- if they even get that far.

There's nothing wrong with having difficult puzzles in a puzzle game; the capabilities, interest, and patience of your players will always span a huge range, so difficult puzzles can keep the best players challenged and give everyone else something to aspire to. However, when progression through a story is blocked by progression through the gameplay, making the game too difficult denies all but your best players completion of the story and the satisfaction and enjoyment that goes with it. By the time we realized this, it was too late. Since our story and our puzzles were tightly coupled, separating them so that the story ended earlier would have required reworking the entire campaign.

The puzzles available in SpaceChem fall into two categories: campaign puzzles and user-created "ResearchNet" puzzles. Although I've heard many players describe their lack of progress in the campaign as "not finishing the game", I've never heard anyone claim that, by not beating every single ResearchNet puzzle, they felt as if they had unfinished business. This seems to indicate that players naturally tag the end of the story as the "completion" point. So, if you're going to make a story-driven puzzle game that gets progressively harder, you really ought to put the hardest content after the end of the story!

We ran into a similar dilemma when deciding where to end the demo. Since SpaceChem is so unlike other games, it seemed fair to make sure that players got a good taste of the game before being forced to make the purchase. The puzzles in SpaceChem are divided into planets (about six puzzles each), with the first two planets consisting entirely of tutorial puzzles.

Ending the demo on the third planet had the benefit of giving players a good feel for how difficult the non-tutorial puzzles were and demonstrated the game's boss battles, the first of which did not show up until the end of the third planet. Unfortunately, it also made the demo about four hours long, which isn't good for encouraging impulse buys.

I've heard claims from various sources (particularly in the mobile space) that making your demo shorter is always better for sales. Even if that's true, I don't entirely regret our decision. Any player who realized the game was not for them when they reached the third planet would have been strongly disappointed if they had to buy the game to learn this. Likewise, any player who made it past the third planet and wanted more would be likely to love the rest of the game. Our business exists to make our customers happy -- why would we act against that?


Article Start Previous Page 3 of 4 Next

Related Jobs

Crystal Dynamics
Crystal Dynamics — Redwood City, California, United States
[08.21.14]

Producer
Cloud Imperium Games
Cloud Imperium Games — SANTA MONICA, California, United States
[08.20.14]

Senior Producer
Yoh
Yoh — Vancouver, British Columbia, Canada
[08.20.14]

Rendering Engineer Job
Yoh
Yoh — Vancouver, British Columbia, Canada
[08.20.14]

Multiplayer Designer Job






Comments


Daniel Campbell
profile image
I honestly thing Zach is under utilized. His genius has brought about some of the most unique and interesting games in recent memory. Notch wouldn't have his fortune without Zach, just saying.

E McNeill
profile image
SpaceChem is, oddly, a game that I feel *guilty* for not playing, yet I still haven't started. I bought it eagerly based on recommendations, I installed it, and I even booted it up once... but I never got past the YouTube tutorial.

I suppose the reason I never went further was just because I saw a wall of difficulty ahead of me (especially after all I had heard from fans), and there was nothing appealing in the core "toy" of the game to pull me in. In most other puzzle games, there's immediately an intriguing story, or pretty art, or a satisfying low-level mechanic to entertain me while I slowly absorb the deeper mechanical challenge. In SpaceChem, the deep mechanical challenge was laid bare, and at that moment I just wasn't in the mood to strain myself to understand it. I don't like what that says about me (hence the guilt), but I guess that's a lesson I ought to apply to my own games.

I still plan to go back to it (eventually), and I'm eager to see what else Zach will come up with.

Bart Stewart
profile image
The "histogram" aspect of SpaceChem deserves much more attention from game designers, as it is responsible for most of both the fun and the frustration of this great game.

Most games do binary challenge reporting: you either succeed at beating the challenge and move on, or you fail and either accept defeat or reload and try again. The fun of this form of challenge measurement is passing a gate by identifying and accomplishing the (usually) one valid solution. Once success is confirmed, you're done with that piece of game content.

SpaceChem's challenge design is radically different. It permits not only multiple valid solutions but offers multiple (antagonistic) scoring systems -- if you think you used too many symbols or cycles (especially compared to the average), you can play that challenge again to try to improve your score. So the fun in SpaceChem comes from optimization within a space of many valid solutions. (Note that while it's not scored so overtly, "multiple solutions" is also a key pattern in the games by Looking Glass and its successors: System Shock, Thief, Deus Ex, BioShock, and the upcoming Dishonored.)

This difference -- between gating and optimization as modes of play -- is crucial for appreciating gamer reaction to SpaceChem.

Gating is simple to understand and rewards persistence. You know exactly what's expected, and you just need to keep at it. For a lot of people, that's fun; it's relaxing to be able to make progress without a bunch of Heavy Thinking.

Optimization is more complex and rewards perceptiveness. Just banging away using the same technique is unlikely to yield a significantly better solution -- persistence alone won't work. To really succeed at optimization, you have to think about the problem, understand it, and generate entirely novel (for you) approaches, often in a burst of creative inspiration. (Programmers will recognize this description as applicable to code optimization, but it's broadly understood as a distinctive problem-solving mode.)

Neither of these forms of play is inherently "good" or "bad." They're just different kinds of fun. But there can be a mismatch between these two forms of fun and the kind of fun that an individual gamer will usually prefer... and that, I suspect, is the source of both the success and difficulty of SpaceChem.

Some people naturally enjoy optimization within a complex space of multiple valid solutions. This rewards their perception and creative thinking, and makes them the natural audience for a game like SpaceChem. They'd have been the ones telling their friends, "You HAVE to play this game!" because there are so very, very few games that are so sharply designed to reward creative optimization-play.

But other people -- almost certainly the vast majority of gamers today -- prefer simple, gated challenges where there's no uncertainty about what to do or how to do it. From their perspective, the world already throws enough complexity at them; "fun" is a simple challenge with a single clear goal condition that can always be beaten if you just keep at it and refuse to give up.

For these folks, SpaceChem is not fun on its face ("ugh, I hate chemistry"), or incomprehensible in the number and variety of action options, or too hard because it rewards/requires thinking rather than doing, or just plain frustrating because there's no single definite "win" state.

So designers are, I think, well-advised to read Zach's excellent post-mortem carefully, especially the histogram part. That's a brilliant design solution for the rare game that creative/perceptive Explorers can love. But "optimizing among multiple valid solutions" is never going to appeal to other kinds of gamers, and in particular is very unlikely to be popular with the masses who enjoy games as simple diversions.

Adam Bishop
profile image
I've got this game from one of the old indie bundles but never actually played it, but this post has made me want to go back and install it, so thanks Bart!

Bart Stewart
profile image
Consider it my long-winded way of saying, "You HAVE to play this game!" ;)

Mond Semmel
profile image
"This difference -- between gating and optimization as modes of play -- is crucial for appreciating gamer reaction to SpaceChem."
I love SpaceChem, and I agree.

But what you mentioned in your quote is not just a difference in design philosophy between different games, but also a "schizophrenic" aspect of SpaceChem itself: As Zach describes in his postmortem, the difficulty curve is too steep. You are always free to go back to tinker with solutions of previously completed levels (the optimization mode of play), but you are always gated from progression until you complete a (hard!) puzzle for the first time (the gating mode of play).

And that hurts the game somewhat. For one thing, solving a puzzle for the first time is orders of magnitude harder than optimizing an existing solution. For another, difficulty spikes often mean that you are prevented from going further in the game, which could allow you to gradually learn and improve. That's very obvious from the completion graph Zach posted. (The defense levels, and the last few levels, are especially egregious culprits of this.)
Often, once I had overcome a "gate", i.e. managed to beat one new level in the campaign, I managed to beat a few more comparatively easily. That also gave me the motivation and new insight to optimize earlier puzzles, which was not only very enjoyable, but also made me learn even more about the game's mechanics.

I completed the game, but I looked at others' solutions to two or three puzzles in the whole game - if you look at the completion graph, I think I looked at the two puzzles shortly before the end (in the categories Flidais and Unknown System) which both only had 50% completion rates among those who managed to reach them. I didn't feel great for looking at a solution, but I still wanted to mark the game off as "completed". (Luckily, due to a strange issue, my exact copy of a working solution of the last level didn't end up working for unfathomable reasons. So I still had some tinkering left to do in the final level, meaning I solved at least part of it on my own.)

SpaceChem is an amazing game, but the difficulty curve/gated content _is_ a very serious issue that hurt the game a lot - especially due to how amazing and enjoyable the optimization aspect of the game is.

JB Vorderkunz
profile image
I MUST NOW BUY THIS GAME! Awesome stuff!

Will Buck
profile image
I was one of those 'questionable' friends that recommended to the engineers I know to try this game. Tons of fun.

I am also one of those people that was 'lost' on the game due to the difficulty curve. That isn't to say I don't still recommend it: I think you get out what you put into this game (hehe), and I put in enough to have enjoyed it a lot and recommend it to others. That said, I never fully groked the predictability of space and time required for some solutions.

As a particular piece of feedback for the devs: The factory-level layout I struggled with, and that piece I feel could've used more training / tutorial pieces. This postmortem is dead-on in strengths and weaknesses, great work Zachtronics team! Inspirational to hear this was a spare-time project for those involved too :)

Keith Nemitz
profile image
Thanks Zach! I really appreciate hearing about the solutions you used to solve various problems. I'm currently facing similar problems with my next game, 7 Grand Steps. It's not a puzzle game, but like SpaceChem, it's so different, videos don't show anything player's recognize as fun.

7GS is about family generations, so the goal is basically to survive. That's a tough goal to tell players who want to strive for something specific. The use of video here, as you tried, may be a good thing. We can use it to explain how earning legends help families to survive.

7GS's core gameplay is simpler than SpaceChem, but I've spent more than one year reworking the tutorial, because a player has to learn TWO rules at the start, to 'get' the fun of the core mechanic. There are a few edge-case rules that can be learned later, but those still complicate the player's model of the core mechanic. After reading your experiences, I think I should rework the tutorial again to offer side-paths that let players explore edge cases.

Thanks, you're a mountain-man blazing a trail for us later pioneers!

Adam Gashlin
profile image
I've enjoyed SpaceChem, bought it day 1 and several times since. As fun as it was to play, it is at least as interesting to me from a design perspective, so thank you for the insightful postmortem.

"We used too much text to explain things."

Yes, definitely. SpaceChem is a model for the flavor of gameplay I want to create, but the text-heavy tutorial screens immediately jumped out at me as a bad idea, even after reading a whole screen of story text! An already running machine with tooltips calling out the new mechanics in use might work?

The walkthroughs themselves might have been better taken in stages, showing what happens differently as each symbol is added. A first level (or sub-level goal) that requires doing nothing but starting the simulation, then one that requires moving the start location, then a turn, perhaps. The jump from that to actually solving problems is pretty big, though. I know how frustrating it can be to keep breaking down concepts, but without being tedious. How much can you expect the player to figure out himself? Those of us who are programmers have a hard time appreciating just how much we assume about the structure of problems and machines.

I want to give the player more mechanisms to manage complexity, rather than require ever more things to be kept in mind, however this doesn't address how to introduce those mechanisms. The "track" that is displayed is very helpful, perhaps more automatic support of that kind is indicated.

One thing that bugged me was boss battles, they seemed to be a complete shift of tone. I haven't finished the main story, as I got to the final boss battle and couldn't bring myself to tackle it, I found the battles so irritating that they weren't enjoyable in their own right. I was surprised to see no mention of these in the postmortem, as I understand they were sufficiently troublesome that they were taken out of the mobile versions. I guess that was more of an interface issue.

Phillip Abram
profile image
I like SpaceChem. I was lucky to meet some of the developers at GDC, nice guys. Can't wait to play their next game.

Patrick Tullmann
profile image
I loved SpaceChem, and occasionally go back to solve some of the ResearchNet problems (though unlike riding a bicycle, I feel like I've lost something if I'm away for too long).

Anyway, despite the Chemistry veneer, I thought this game was a fantastic gamification of programming concepts. Specifically of multi-threaded coding and synchronization. The ability to build lock-free (well, 'sync free') solutions to many of the puzzles was a personal goal of mine.

I too found the tutorial under illuminating, but in its defense, there is a lot more reward for figuring SpaceChem out, since you're clearly not being hand-held through the solutions. For example, the concept of a "waldo" made no sense to me. But, once I'd gotten through enough of the puzzles and had enough "ah-ha" moments, I could go back and re-do some of the earlier solutions. I found it particularly gratifying to look back on my original solution to an early puzzle and realize how much I'd learned since then. Where originally it had been opaque and difficult and more of a trial-and-error solution, now I could see how "easy" the puzzle was.

Muir Freeland
profile image
This article convinced me to buy SpaceChem. It sounds awesome.

q groozl
profile image
I'm one of those who bounced right off it. I think the postmortem is pretty accurate in what was difficult about the explanation.

However I challenge the notion that the game makes sense to developer types. I've been a professional programmer for going on 1.5 decades now, and I never liked games that are programming analogues. Designing algorithms with extremely tight limits on variable names and problem decomposition just feels like a puzzle on how to write a program in the worst possible way.

Maybe EE types wouldn't run into this since dealing with real constraints is part of their normal world. But I always think "where's my vi session and debug log? Why can't I create my own functions?"

Adam Gashlin
profile image
Yeah, kind of like implementing machines in Minecraft, Little Big Planet, or Conway's "Life", it's a matter of being amazed that you can manage to do anything. The things you ultimately end up doing aren't terribly interesting (though the boss battles may be an attempt to work around that). That said, it's a fantastic puzzle of that type, for those of us too lazy to find our own challenges in existing sandboxes.

Back when I started playing SpaceChem I wrote a bit about this, in the context of games for learning programming: http://blog.gashlin.net/?p=415


none
 
Comment: