There are two main ways I come up with game ideas: story first, or mechanics first.
When I come up with a new, interesting mechanic, I think about what genre that mechanic will slide into most easily, and once I have that, the rest of the game, from story to levels, arises naturally. Based on the mechanic, I might design a shooter, a platformer, or even a board game.
When I come up with a story first, I always make RPGs.
This is kind of an unfair statement, since my process for story is a little different than for mechanics.
When I design mechanics, I figure out which game genre it fits best with. When I come up with stories, I figure out which medium overall it best fits into, from short story or novel to play, movie script, musical, song, poem, or game.
Once the story medium has been narrowed down to game, only then does RPG seem the most obvious choice, and rarely do other genres come to mind.
I think this is partly because RPGs are the most similar to other mediums. They follow the hero's journey, and allow for lots of dialogue, backstory, and complexity without bogging down the game. In fact, those things are typically expected by players of the genre. What is bad in one genre works great in an RPG.
There is a reason traditional stories work best in the RPG format, and it mostly has to do with the separation between story and gameplay. Story takes place in cutscenes, dialogue, scripted events, and artifacts (such as tomes which give backstory). The gameplay itself involves exploration and battle, mostly. Battle is no more than an action scene in a movie, and what matters is whether you win or not, not how you get there.
When I posted an article to Gamasutra about why Games Aren't a Storytelling Medium (Yet), I got a lot of comments and little bit of criticism of it. One comment in particular mentioned that story in a game is not the same as story in any other medium. The story of a game is the minor actions, or the gameplay.
Take Half-Life, for example. It's an example of the finest in linear first-person shooters that have ever been made. Story-wise, nothing changes from playthrough to playthrough. That is, if we are only looking at the traditional story--the story that would matter in another medium. Accident at facility, aliens invade, military comes in to stop it and kill scientists, Freeman saves the world.
But no two games of Half-Life are alike, because each set piece action sequence can be played an infinite number of ways. Our silent protagonist's personality is defined by player action, not by story. Gordon Freeman is a frightened scientist, untrained in gun combat, and unsure of his every move. Or Gordon Freeman is cool under pressure, confident in his abilities, and is an all-around action movie hero. He can be either of these things, depending on how the player chooses to play the game.
The traditional story happens around Gordon Freeman, and Gordon Freeman is You.
In the end, the story that matters in Half-Life is not about an alien invasion, and it's not a strictly linear story. It's about how you, the player, construct the personality of the blank slate we arbitrarily call Gordon Freeman. And in that way there are an infinite number of stories. Maybe Gordon begins unsure of himself, but his confidence grows as he gets used to the action. Maybe each new twist upsets him; maybe he takes it in stride. Maybe he's afraid of the aliens, but not of the soldiers; maybe the opposite is true.
In this way, games aren't a storytelling medium--they're a story making medium. The player creates the story him/herself, and the traditional story of the game is just the wrapper the developers made to allow you to experience new situations.
But not all games have blank slate heroes. Some have very clearly defined protagonists, like RPGs, or God of War, or Metal Gear Solid. You see the protagonist's personality in the cutscenes, and you are given little wiggle room to put your own personality on top of it, and hence, control the story.
Sure, in an RPG, when you enter battle, you get to decide the proper tactic you want to use, and you get to decide what sub-quests to take, and heck, in some games of many genres, you even get to decide which ending you want to see.
But in most of these games--pretty much any game with a clearly defined protagonist--the traditional story is front and center, and what the player chooses to do is of little relevance. It no longer matters how the player wins a battle, only that the player wins the battle.
So when I come up with a story idea, I am by nature or habit coming up with a traditional story, and therefore an RPG is a natural form to follow. Other forms work as well, but an RPG is the most obvious, and has me think least about how the game will function.
Some games blend traditional story with player agency well, balancing them in a way such that the player is being told a story and is making a story as s/he goes.
Take Silent Hill 2, for example. The traditional story is heavy, and the protagonist appears to be clearly defined, but your style of play determines the final boss battle and ending. You don't consciously select A or B; instead, the actions you take, how you react to a new monster, how you react to a new item, how you react to your health falling, how you react to other characters; all are taken into account behind the scenes, and then you are presented with an ending that best represents the protagonist you created over the course of the game.
Older games, before story was a big deal, were entirely about player agency, and the player created the story through gameplay, through chosen action. The original Warcraft and Wacraft II were about the player's actions, because the story was barebones, and what mattered was how you defeated your enemy, not that you defeated your enemy.
As games as a medium grew, they tried to push in more traditional story into the games. Warcraft III is far more about the story of the characters, which you have no control over. Arthas will become evil because you played the game, not because you wanted him to become evil, and not because you made him evil through your gameplay decisions. The story of Arthas is told to you through scripted events, and you're just along for the ride.
Yet they have nearly the same gameplay: they're both RTS's, and Warcraft III is only advanced in an evolutionary way that one would expect, but they are still basically the same.
The difference between them is their emphasis on the player's created story versus the writers' traditional story.
In an inarticulate way, I've always held this vague notion of the difference between the two forms in games. I used to be content with either style of story, and would gladly watch a cutscene that was twenty minutes long, or just as gladly play a "storyless" game like Combat for the Atari 2600.
But as I get older, I have started to become old and crotchety and want my games to allow me to control the story, and make me, the player, matter. I tend to think "If I want a traditional story, I'll read a book."
There are plenty of games that provide both kinds of story at once and give equal time to them both (and, in the aforementioned example of SH2, allow one to effect the other), and these have the best of both worlds.
But ultimately, as I get set in my ways, when I play a game I want to play it, not watch it, so I skip cutscenes, rush through dialogue, and just get on with the gameplay.
Maybe that is my fault, as I have said, or maybe story writers need to think more carefully about what the best medium is for the story they want to tell. After all, I may be a game designer, but when I come up with a story idea, I don't shoehorn it into a game if I don't have to. Maybe it makes a better book.
Sometimes the opposite problem arises, I think: the traditional story is an afterthought in the development, and anything will do, as long as there are a few cutscenes in there, because there "must" be a story these days.
If a traditional story is truly necessary (and I don't think it is, but that's beside the point), then the story needs to be something that either naturally arises as a consequence of the genre and mechanics that are being used, or the story must at least be designed in tandem with the mechanics, and more care must be taken to marry the two.
Because if the traditional story and gameplay don't flow together, it becomes jarring when one interrupts the other, and flow is broken. I think that's the last thing a game developer wants to do.
To read more article likes this (with pictures and jokes), or development logs, reviews, and more, check out http://scattergamed.blogspot.com/