Let's Talk About Touching: Making Great Touchscreen ControlsBy tim rogers
A reprint from the January 2013 issue of Gamasutra sister publication Game Developer magazine, this feature explores touchscreen control methods. You can subscribe to the print or digital edition at GDMag's subscription page, download the Game Developer iOS app to subscribe or buy individual issues from your iOS device, or purchase individual digital issues from our store.
If I am not mistaken, you are not six years old, so it's fine if you object to the following claim:
Buttons are doomed; touchscreens are the new game controllers.
I've been working on making mobile games as the founder of an independent studio called Action Button Entertainment. In order to make the best mobile games, I've been dissecting and researching every interesting game-control mechanic I can find, from Pong to Angry Birds. Here is what I've found.
I've always been a proponent of the mechanical particulars of a game's feel over any sort of gimmick related to its product construction. The triumph of Super Mario Bros. was one of Game Design by the Milliseconds -- of the developers pre-understanding the game as more than a series of short-, medium-, and long-term goals. Super Mario Bros. is about the immediate-term goals, and the way that the player's microscopic actions feel in increments of five or six milliseconds.
Super Mario Bros. felt like magic. The fine degree to which the minute variations of button-press length could affect Mario's jump heights and lengths checked every box in my child-brain's "best thing ever" wish list. More than 20 years after the Nintendo Entertainment System, we had the Nintendo Wii: Shake a little television-remote-like thing any which way to make a cartoon person hit a tennis ball. Nintendo was shifting the emphasis, aiming for the part of the brain that makes a six-year-old find a television remote control more magical than Asteroids.
Games With Buttons are not superior by default; they are superior because a parade of geniuses like Shigeru Miyamoto laid the groundwork. Players needn't wage a culture war of casual versus hardcore, social versus everything else, mouse and keyboard versus controller; in a perfect world, action gamers would only speak scientifically of the milliseconds of a game. If a game's milliseconds unite its action and its player, then the game is real and true.
In order to understand the touchscreen-versus-button dichotomy, let's revisit the old mouse-and-keyboard-versus-controller debate: You can't click on a recent blog post about or review of Halo 4 without accidentally scrolling down to the comments and seeing someone groaning about how they'll never play a first person shooter on a console because "mouse and keyboard is the only way to play an FPS." This opinion has raged since the moment Halo was announced as an Xbox exclusive.
I am convinced that we could get a room full of theoretical physicists to prove that Halo does a pretty darn good job with a controller, and that Call of Duty's by-the-millisecond design concessions for controller players (such as the smart snap-to auto-aiming) add up to a game that would be just as competitive with a controller as it would be with a mouse and keyboard -- that is, if all the best first-person-shooter players weren't born and raised with a mouse and keyboard in their hands. As the mouse disseth the controller, so does the controller disseth the touchscreen. (And we all diss motion controls, but that's a topic for another day.)
And maybe most touchscreen games deserve the disses.
I see a lot of games with virtual buttons on the screen. This is always a mistake. That's a sign of a subliminal conspiracy between game developers -- everyone on the team silently agreeing to commit to an inferiority complex: "[Sigh.] It sure would be cool if we were making a game with buttons."
Know this: A friend was telling me just the other day that his four-year-old son just doesn't want to touch a game controller. This friend has a glorious collection of old and new game consoles in his many-televisioned house, and all his son wants to do is play Where's My Perry? on the iPad. "Controllers just aren't real games to him," he told me.
I know "hardcore gamers" that will spend an hour of their lives trying to make a Skyrim avatar that looks exactly like themselves, and then they'll say mobile games aren't "real" games because their fingers get in the way of seeing the screen. This is interesting. Personally, I prefer IMAX to 3D movies, because I like feeling like I'm inside the movie, rather than feeling like the movie is popping out at me.
Now, imagine the way a four-year-old child feels playing with a touchscreen: The child touches her fingers to the screen, and the simulated world reacts. The child can literally touch her favorite cartoon character, and watch that character move. How is that not superior to pressing a button over here and watching the character move inside that screen over there?
Modern touchscreen technology has closed the distance from which children will consider electronics magical. For a four-year-old -- one who, in 10 years, will be a 14-year-old buying the games you're hopefully still making -- your remote control simply won't cut it.
Patient Zero: Pong
Designing essential game mechanics for touchscreens requires an understanding that hardcore action has never, ever been "about" the control method. It's about the way the action interacts with the player's brain. The control method is only ever an instrument for fabricating that brain-screen coordination.
Let's consider Pong. It's a hyper-competitive, finely nuanced contest between two players. The control implements are nothing like modern video game controllers. Players twist a tiny, mosquito-bite-sensitive knob. Twist a tiny bit clockwise, and your paddle zips to the bottom of the screen. Twist a tiny bit counter-clockwise, and the paddle zips to the top of the screen.
Furthermore, the paddle is made up of eight segments that appear to be seamless: The part of the angle of the return depends on where the ball contacts the paddle. Throughout a Pong contest, the player must balance the urge to trick their opponent against the urge to fire a return which is not so tricky as to result in any tactical backfires if returned. I like to think that the essence of all hardcore action games is purely available in Pong.
What I most take away from Pong is the relationship between the paddle and the control knob: The delicate sensitivity creates a brain impression that the game is more than just something happening on the screen. It's an object with a physical presence on planet Earth.
That brings us back to the image of a child, holding a screen, touching her favorite cartoon character, and watching that character react, like magic.
Case Study #1: Jetpack Joyride
You might have played Jetpack Joyride. It's a casual, free-to-play, monetized, mobile game -- all those buzzwords -- though it's also an action game, and its central action is tuned to a level of nuance on par with any individual gun in Halo.
In Jetpack Joyride, the player touches the screen -- anywhere -- to fire the character's jetpack. The longer the finger contacts the screen, the longer the jetpack fires. A short burst and then release will move the player-character less than halfway up the screen. A longer hold will bump the character's head into the ceiling. To obtain skill at the game, the player must master tapping and releasing as obstacle-context requires, juggling the character eternally between positions that are neither precisely the top nor precisely the bottom of the screen.
Jetpack Joyride is tenaciously feel-balanced. The speeds of by-jetpack ascent and by-gravity descent are locked in a tastily lopsided yin-yang. Jetpack Joyride is what happens when a game gets even one thing truly, actually, perfectly right: millions of downloads.
I first played Jetpack Joyride as someone who had tremendously enjoyed Canabalt, the pioneer of the "endless runner" genre. Canabalt is another game that gets one thing truly -- actually, perfectly -- right. Canabalt is about tap-hold-duration-sensitive jump heights with inevitable gravity. It's about the beauty of parabolas. (Action games are almost universally about parabolas.)
As your character accelerates, his maximum and minimum jump distances grow; the shape of his minimum jump parabola becomes an oddly stretched caricature of its original self. The game becomes "unpredictable" until you master the feel of it.
Canabalt and Jetpack Joyride are games that most players have experienced exclusively with a touchscreen, though they are also highly playable with a mouse -- as their Flash and Facebook versions, respectively, indicate. Therefore, it's probably better to call them "one-button" games; they work well on a touchscreen, since a touchscreen is a big button, but touchscreens do not have to be one-button games.
In all sincerity, I say that Canabalt and Jetpack Joyride, two games whose creators I presume love Super Mario Bros., have actually demonstrated a masterful understanding of that game's playful friction. But is it possible to make a one-button Super Mario Bros.? By god, no.
Super Mario Bros. is not just about the fine manipulation of jump parabolas through button-press durations; it's also about direct manipulation of running speed. When I was a kid, you knew another kid knew what was up in Super Mario Bros. if he was holding the controller perpendicular to his torso, buttons out, thumb keeping the B button depressed.
Narrating the experience of parkouring through World 8-3 is no easy task: It'd be a Morse code of taps and releases of the A and B buttons, with B held down most of the time and only let go for 15 to 20 milliseconds at a time as obstacles or enemies demanded it. Super Mario Bros.'s mechanics are deceptively rife with moving parts, and there's probably at least two whole textbooks to be written about the friction when Mario slides to a stop before changing direction.
If we remove the ability to change direction from Super Mario Bros., and make it a two-button level-based runner, where one button is "continuous horizontal thrust/acceleration" and the other button is "jump," well, now we're getting somewhere. Touch your left thumb to the left side of the screen to make your character accelerate; release to decelerate. Touch your right thumb to the right side of the screen to jump. (I can't think of any mobile games that use this scheme, though if you've made one, please tell me -- I'd love to play it.)
Now we just need to super-fine-tune the acceleration rate, the top speed, the deceleration rate, the jump-height acceleration, the jump-height cutoff, the speed of gravity, and craft interesting multiple-level paths and enemy behaviors. Do all that, and you've made something pretty Super-Mario-ish without using more than two buttons or a touchscreen.
Of course, this two-button Super Mario Bros. would also be about 27 Rubik's Cubes-worths more complicated than Canabalt and Jetpack Joyride duct taped together. It'd be like Canabalt: Stick-Shift Edition. Developing it into a successful product would be a nightmare (if not multiple nightmares).
Case Study #2: FASTERBLASTER
During the prototype phase for my studio Action Button Entertainment's upcoming hyperkinetic touchscreen mindsport FASTERBLASTER, I hit a brick wall in my attempts to explain the way the player's avatar should rotate.
The avatar is, at the end of a movement, always pointing upward; the device is always in portrait mode; the camera is rotating along with the avatar.
Our second prototype made us seasick, because the camera rotated at the exact speed as the avatar, and the avatar's direction related one-to-one with the position of the player's finger on the bottom edge of the screen. To return to the Pong example, it would be kind of like tweaking a Pong knob one millimeter counterclockwise, and then finding yourself suddenly standing in the arcade parking lot.
Only enemy formations could lend context to FASTERBLASTER's grotesque spiral. I had this quirky idea that the player's finger-touchdown point should correspond directly to a clock-face position. Assuming the avatar begins pointing to 12 o'clock despite no touch input, touching the left edge of the screen would point the avatar to 6 o'clock. The right edge, also, would point the avatar to 6 o'clock. Now, the very center would point to 12 o'clock; left-center would point to 9 o'clock, and right-center would point to 3 o'clock.
In FASTERBLASTER, the player charges up grenades and then lobs them at enemies who are constantly advancing from all directions. The grenades' destination point is constantly fluctuating between the tip of the triangular avatar and the upper edge of the screen. Each up-down pump increases the charge level of the grenade, which increases the radius of its explosion if it contacts an enemy.
So it's important to be able to hold onto your bullet while it's charging and shift your aiming position easily. The bullets charge quickly, and the target cursor moves quickly. The enemies move quickly, and the travel time of the player's grenades is balanced to enforce maximum friction.
It's just -- that sliding. Lord, it just about gave me an aneurysm.
Part of the problem was the size of the iPhone screen. The ratio of my thumb-tip-width to total width of the screen in portrait mode to circumference of the playing field (in iPhone screen heights) was hardly golden. An iPad Mini (the target platform) is just fine, because the travel speed of the average person's finger across that real estate yields a sensible enough rotation speed. On an iPhone, though, forget it.
My pride cried. I'd believed in the FASTERBLASTER pre-prototype controls, because they'd worked so well in ZiGGURAT.
Or had they?
Case Study #3: ZiGGURAT
I had coffee with a friend several months after we released our debut game ZiGGURAT. His website had spoken favorably of the game, though, as my friend admitted, he "didn't really get it." I asked him to open the game on his iPhone and show it to me. He complied. "Give me your best performance." There it was: He was just tapping his finger all over the screen. "I can't see the guys," he said. "My finger gets all over the screen."
Of course, he'd skipped through the (merciful, exceptionally brief) tutorial. So I showed him how to play: Slide your finger along the bottom of the screen. Touch the middle, and the character standing on top of the pyramid points straight up. Touch the right, and the character points down and to the right. Touch the left, and the character points down and to the left. This way, you can slide to aim your shot.
Some reviewers had said the game was "Just like Missile Command." These reviewers had probably never played Missile Command. (Note: I love Missile Command.) "The game would be better with buttons," another review had said. Not that I cared about those reviews; EDGE gave us a 9 out of 10, so whatever. The whole point of ZiGGURAT had been to make a game that was for touchscreens.
In ZiGGURAT, you hold your finger on the screen to charge a bullet. The bigger the bullet, the straighter and faster it flies. Bullets grow and shrink as they charge. You'll have to deeply understand the feel of the charge timing to get the parabola you want.
We designed eight unique enemy types (many of whom are constantly jumping up and down) to keep the parabolas maddeningly varied and thus interesting. And the enemies' heads are growing and shrinking at a rate that's just different enough from your gun-charge speed to defy muscle memory: The bigger the enemy head, the bigger the bullet, the bigger the explosion, the bigger the chain reaction, the longer you stay alive, the higher your score.
At the center of all this is the player's ability to always be in control of exactly where the gun is pointing. You're not just charging and releasing shots, you're also aiming the gun left and right, and every point your finger travels through correlates exactly to a position the gun can be pointing.
Moreover, the bottom of the screen is a negative space: the silhouette of a pyramid. Your finger doesn't get in the way of anything, and the "control pad" on the screen is both contextualized and practically invisible.
I feel pretty clever for thinking of all that: I wanted to do for a first-person shooter what Canabalt had done for Super Mario Bros., except we had to stop at Japanese 1980s arcade games and StarCraft along the way.
Not everyone loved the controls. A couple of iTunes reviews were livid at the lack of "offset controls." They wanted an invisible virtual analog stick wherever they put down their finger. I feel like this would have ruined the game: They'd get what they want, and then, unable to aim in literally every angle, their scores would suck, and they wouldn't get it, and then those emails would start coming again.
Case Study #4: Mac OS X
Taking ZiGGURAT's control style over to FASTERBLASTER was a nightmare. Luckily, I had a backup plan. I feverishly typed a long email to programmer Michael Kerwin, in which I explained controls that'd work "like an old-fashioned stereo knob except not really." He said he'd need a couple hours to think it all through.
A couple hours later, he came back with the suggestion that the controls I mentioned were "like the iPhone alarm-setting wheel." "Oh," I replied. "Yeah."
If you ever meet my parents, they can confirm this: I have, since childhood, had perhaps unnecessary amounts of fun with unlikely things. Recently, one of those things is the multiple desktop switching on the more recent Mac OS X versions. Sometimes, I'll put four fingers on the trackpad and record-scratch between two spreadsheets in time with the music I'm listening to.
The best thing is, of course, "natural" scrolling in OS X. I just spent 30 seconds scrolling up and down through this document. This scrolling possesses such charismatic friction in its coasts and turnarounds. It's as finely honed as Super Mario Bros., but it's smooth in its nuanced complexity to allow for perfect natural usability.
Natural scrolling accelerates to a set point relative to the speed over distance of a two-finger swipe. To halt the scrolling immediately, just touch one finger to the trackpad. I have watched many Mac users scroll and halt, and they always use two fingers for the halt, even though one finger will do. This is important: "Two fingers" is attached inseparably from any action related to scrolling. This is as close to "proof" of an interface's intuitiveness as we can get.
The most important aspect of natural scrolling is the natural deceleration. Somebody working at Apple loved that action. OS X's two-finger trackpad scrolling is to a mouse scroll wheel as Netscape is to Google Chrome. So here's where we have to talk about "friction": Without some quirk to snag an action and pull it away from "perfect usability," an "interface" cannot become a "game." The soul of game design, after all, is in assigning rules to objects.
Natural scrolling in OS X halts immediately when I put a finger back on the trackpad. What if the finger on the trackpad were brakes? What if to halt the scroll, I had to input a reverse of the gesture -- matching the speed over distance of my previous swipe, only in the opposite direction? Now we might have made a game: You have 10 seconds to scroll the scroll bar up to a touchdown zone; you have to land it precisely between two lines. Swipe up to accelerate. Swipe multiple times to accelerate higher and faster. Now keep your distance over time in mind when you start applying the braking swipes.
We've just started designing the Per-10-Seconds experience of a stupidly abstract train simulator or an Atari-2600-like curling game. It's probably not very fun. That's okay, because we're not actually going to make that game.
Instead, imagine you have your smartphone in landscape mode. Now imagine we're making Super Mario Bros. Your character is facing to the right, and located just a bit left of the middle of the screen.
Slide your thumb up and down on the left side of the screen to move right and left. Double- and triple-swiping dials in multiple accelerations. You're controlling Mario with a scroll gesture: Swipe X distance in Y time and then release to accelerate Mario up to top speed for Z seconds, after which he starts to slow down again. Swipe X distance in Y time and then leave your finger down to keep Mario at that top speed. Touch (and hold, and release) anywhere on the right half of the screen to jump.
Now try prototyping this. So, uh, why do people use virtual buttons, again?
Case Study #5: Angry Birds and Gasketball
You could play Angry Birds with a keyboard, just the same way we used to play Scorched Earth on a PC. The up and down arrow keys pitch the angle of the bird, left and right pull back the slingshot, and the space bar fires. But by being a touchscreen game, Angry Birds offers children of all ages the joy of reaching out and literally touching their game character with their finger. That slingshot-stretching sound enhances the perception that our flesh is making something happen. It's barely tactile, but it's still tactile. The world inside the touchscreen is a little cartoon still life of a pinball machine; we wind it up and watch it go.
I'm not here to talk about Angry Birds' innovations or branding or level design (which all sort of suck, to be perfectly honest): Let's just say it feels like something. It's not something that we necessarily have to be touching, though it is popular as something that we touch.
It's safe to say that Angry Birds would not be popular without either its branding or its mechanical particulars. Angry Birds is as much a house of cards as any hundred-million-dollar property. My opinion, though, is that the controls and mechanics are as sound as the characters are ugly, and that without the surrounding game design the birds would be dead on a street corner somewhere. (I mean, they don't even have wings.)
I wouldn't be surprised if one Angry Birds-esque stuff-throwing "clone" is released on the app stores of the world every six hours, but only one stuff-throwing game stands out -- and this is a perfect way to end what I'm trying to talk about in this article.
Gasketball for iPad is a game about throwing basketballs into basketball hoops. On the way to the hoop, your ball must bounce off of various objects or walls. You must sink each stage's basket challenge in one throw.
The speed of the balls' flight through the air is faster than four of the angriest birds taped together. The physics are deliciously swift. Yet the controls are not a keyboard-surrogate touch-slingshot. They put Angry Birds on Xbox 360, you know. You can play with just an analog stick. It's actually intuitive and highly playable.
Gasketball would be neither intuitive nor playable at all with an analog stick: In order to shoot the ball, you trace a swooshing pattern, putting spin or speed or power or all three behind it simply with a flicking flourish of your finger. The range of expression contained in a single 0.15-second input is astounding.
Once you've beaten one "world" of stages -- the game gives you limited balls; lose them all, and you start the set over -- you unlock the ability to play a time-attack mode, and here's where my brain lights on fire. It's like Angry Birds, only you are angrybirding for your mortal life, at reckless speeds, applying full labyrinths of brainwork to every twitch of the finger.
This is what a touchscreen game is: a single simple joyful motion pregnant with consequence, birthed into a stage where things are happening. You must time your action to fit between, around, or through obstacles.
In short, touchscreen action games require incredible feel in simple actions on the tens-of-milliseconds scale, and superlative level design. Hey -- that sounds like all other action games.
Call of Angry Birds
Gasketball is as "real" a game as Call of Duty (at the very least).
I first learned about Gasketball from Bennett "QWOP" Foddy, after showing him our tennis-like ball-and-paddle game TNNS (iOS/Android). In TNNS, aftertouch is everything; you tap the baseline to move your paddle, then slide or tap your finger along the baseline during the 0.xx-second freeze-frame impact after the ball hits the paddle to bend the ball's flight path. The bend is relative to the distance from the paddle (and ball) at which you released your finger, and the time it took to travel there. You can change the direction twice -- tap twice on opposite sides of the baseline to do freaky right-angle bends. It's fun for multiple players (and it has a single-player mode, too!).
After showing him TNNS and getting his approval ("TNNS is great." - Bennett Foddy, Oxford University), I told him we had something more fun in mind for a golf-like game. I explained it, and then Foddy said I should try Gasketball. I saw one YouTube trailer, and almost chewed a row of teeth-holes in my bottom lip. Well, great minds think alike, et cetera. It just might be that some of the Great Touchscreen Games of the Future are here already, and more are arriving as we speak.
Return to the full version of this article
Copyright © UBM Tech, All rights reserved