Class design as market inefficiency
In a perfect world, universal specialization leads to maximum productivity for the economy. From a game design perspective, there are two things wrong with that statement.
First, we don't want everyone to specialize. We want both specialists and viable hybrids that can serve in two or more roles. There are relatively few primary roles, and unless we have viable hybrids, the game is going to end up being extremely simplistic.
Secondly, we don't want maximum productivity. We want whatever level of productivity matches the intended challenge level of the game with the most fun group dynamics. From a certain point of view, a game designer's entire job consists of putting barriers in the players' path. They're supposed to be interesting barriers, obstacles that provide flavor and enjoyment, but they're still barriers.
The key phrase here is "in a perfect world". The economic laws mentioned above are predicated on certain assumptions, and as long as those assumptions hold true hybrids will experience strong implicit pressures to specialize.
Therefore, the essence of hybrid class design becomes violating these assumptions to create an imperfect world. A perfect party economy is a boring party economy, with everyone forced into fixed roles and no nooks and crannies for hybrids and unique classes and playstyles.
Not all of the assumptions have to be broken in order to create a viable niche for hybrids, but multiplayer gameplay is messy and we can't rely on any one trade barrier to stem the pressure that naturally destroys hybrids. Players find ways around the limitations we build into the game mechanics, so the more redundant barriers we include the more likely we are to have the effect we want.
Breaking assumption #1: Homogeneous output
One of the fundamental assumptions of the comparative advantage model is that a commodity is the same no matter who produces it, that a bushel of wheat from one country is equivalent to that produced by any other. In MMO terms, this would mean that all healing or tanking is the same, even when produced by different classes.
If all tanking is the same, then all tanking classes can be directly compared to one another and one will inevitably come out on top. This would lead to a pressure on characters in the optimal tank class to specialize in tanking and all other tank-capable characters to specialize in other areas.
We can break this assumption by offering different "flavors" of the three basic commodities. If one type of tank is better at absorbing magic damage, while another is better at absorbing non-magical damage, then we have split the single commodity into two commodities, which can be supplied by two classes. The more commodities, the more potential for hybrids who can fill multiple niches.
Just creating the flavors in the basic game mechanics is the first step, but it must be supported throughout the game by ensuring there is a demand for all flavors. Making one type of tank better at absorbing hostile magic is pointless if there is not sufficient hostile magic out there to be worth tanking. To put it back into economic terms, non-homogenous output must be matched by non-homogenous demand.
Breaking assumption #2: Free entry and exit from the market
Another assumption is free entry and exit from the market, the idea that producers are entering the market when it's to their advantage and leaving it when they no longer benefit or can't compete. In an MMO, this idea manifests as swapping characters in and out of a group for different tasks.
If the group is going to do a fight that requires extra healers followed by a fight that requires extra tanks, they can achieve maximal efficiency by adding extra healers to the group for the first fight and then swapping them out for additional tanks for the second fight.
This is a perfect example of a potential hybrid (a healer/tank) being displaced by two specialists. Because the specialists can enter the market freely when it's convenient and leave when it's unprofitable, they can out-compete a hybrid who must divide their strengths between two roles and stay in the market full time.
The City of Heroes Task Force system is a great example of a game where the design does not allow free entry and exit from a group. A Task Force is a series of linked missions, and the group of players present at the start of the task force is locked in to the task force. Players can leave the task force, but new players can't join.
This sort of lock-in produces a counter-pressure to the natural specialization pressure, encouraging players to take a more balanced approach. As with any design decision, this counter-pressure comes with a cost. Players like being able to log off when they want to, and may be less likely to join a Task Force if they know they are locked in until the end. Also, the hordes of newer, casual MMO gamers who have started playing in recent years are unlikely to appreciate a game which is unforgiving of real life interruptions.
An alternative model is the World of Warcraft raid ID system, where players can only participate in a given raid once per week. If a raid group swaps in an extra DPS player for a single fight, that player is then barred doing that raid again for the rest of the week. This potentially could make the player or group think twice about using up the entire potential week's contribution of that player for a onetime benefit.