Dialogue trees are a vital part of many role-playing games, and similar styles of dialogue are used in other games as well, especially adventure games. The primary idea behind dialogue trees is that the player should be allowed to decide who the main character is, rather than the designer. However, there is one major problem that this approach often ends up creating - the resulting characters have no character.
What Is The Problem, Exactly?
The crux of the problem is this - when creating a game like Fallout 3 or Dragon Age: Origins, the idea is that the game should not define the main character for the player, since the main character is for all intents and purposes supposed to be the player, and the player knows better than the designer how they want to respond. However, because of this, the main character is almost never given anything to say; instead, the character is only allowed to ask questions.
This means a couple of unfortunate things. The first is that the player is never actually the driving force of the conversation (and, as a result, never the driving force of the story). While it is true that the player is often allowed to choose how the story plays out, they are only able to do so within the boundaries set by NPCs. It is always the player who responds to the NPC, and never the NPC who responds to the player. The story is never about what the player's character wants, it is always about what NPCs want out of the main character.
Because players are not allowed to be the driving force behind in-game conversations, dialogues virtually always take the form of question and answer sessions. The NPC has information the player wants, the player knows (or believes very strongly) that the NPC has this information, so the dialogue becomes primarily about how the player can extract the information they want from the NPC. The player simply asks the NPC questions in order to discover information that will help them solve a problem. That is why I have called this The Interrogator Problem, because the player essentially plays their character as an interrogator.
Perhaps the most unfortunate result of this dialogue structure is that the main character winds up with virtually no personality. This makes for some very uninteresting story-telling. One of the most basic rules of story-telling is that everyone must have motivation, or more simply that every character must want something. More complex and important characters generally want many, sometimes conflicting, things. However, player characters are only allowed to want one thing, which is information that will bring them closer to the game-appointed end point. But this isn't really the motivation of the character, so much as it is the motivation of the game.
What does the player character believe? What do they value? What do they think about? What do they want from the world? From other people? From themself? The lazy answer is "whatever the player decides", from which we get my most hated video game trope: the silent protaganist. But the real, honest answer to all of those questions is nothing.
Why Does The Problem Exist?
Like many of the problems we have to deal with in game creation, two of the biggest hurdles are technology and time. It seems like it would take a huge effort to create the breadth of content that would allow players to actually direct the story, rather than merely responding to it. Particularly in a game in which there was voice acting, this would seem to be a huge barrier to overcome.
Another important reason that the problem exists is that games are almost always organised into quests. And quests, it would seem, must follow a particular structure - someone has a problem, the player would like a reward, and so the player solves that problem to obtain said reward. But perhaps it does not need to be that way.
How Do We Let Players Make Better Characters?
The most important answer to this question is actually contrary to the question, and that is to take some control away from players. We can allow players to choose the direction that a conversation will take, but we can also write dialogue that we think is fitting for that direction. If the player chooses to say "No, I could never kill an innocent", then instead of having an NPC immediately respond to that brief remark, the player character could then provide an explanation of why they would never kill an innocent.
Yes, it's entirely possible that the player will say "Hey, that's not actually why I could never kill an innocent!" But isn't it better for the game to have the character say something that's at least interesting than to say nothing at all? And perhaps just as importantly, players may also be quite pleased to see that their character is actually a part of the world rather than simply a witness to it.
Another way to do this is to simply expand the way in which we think of dialogue trees. Normally dialogue trees are set up to fit a morality system or a skill system (thieves trick NPCs with cunning, warriors intimidate NPCs with strength, wizards use logic to convince NPCs with intelligence, etc.) But dialogue trees can also take the form of different opinions. For example, what if instead of good/bad/neutral morality, we let players choose left/centre/right politics? It makes perfect sense for a character to give a detailed and personal explanation of their political standpoint.
There are plenty of other ways dialogues could be set up: upper/middle/lower class, atheist/agnostic/devout religious views, humanistic/scientific/religious world views, etc. There are many ways to provide players interesting options while still providing a way for their characters to have more to say than simple one line questions/answers.
Another step that we could take is actually very simple - turn the NPC into the interrogator and the player into the one with the answers. Dragon Age does this to some extent, where other characters in your party will ask you how you feel about something, and then ask you to explain your answer. I personally thought those portions of the game were among the more interesting dialogues to play through.
However, those dialogues were almost always brief one-offs that had little do with the game on the whole. It is a great way for the player to feel as though their character is larger than just the main plot, and so in that respect those dialogues are exactly the kind of thing I think we need more of. But it would also be great if those sorts of dialogues would occur during the main storyline as well.
For example, in one part of Dragon Age (minor spoiler alert)(edit: originally typed Dragon Quest, for some reason) the player is tasked with choosing which of two warring dwarven nobles will become the next king. The player does this by completing a quest line for one or the other. At the end, the national assembly asks the player to crown a king. What if instead, the potential leaders all had distinct political views, and the player was asked not directly to choose a victor, but to give a speech to the assembly in which they had to choose a set of political views to support. If the assembly was sufficiently swayed, then instead of asking the player to choose a king, the assembly would select the leader who they thought best embodied the opinions that the player character had elucidated. In this way the player is allowed to play an important role in the story not just by making a selection, but by having their character actually represent that selection in a tangible way within the game world.
The most important thing to do is to allow player characters to say things which are more than brief questions targetted at NPCs. That way players will feel as though they are driving the story, and not just witnessing it. So that's The Interrogator Problem, and those are some of the most promising ways I can think of to fix it.