What goal do you keep foremost in your mind when you're programming AI?

It depends on what the function of the AI is. If it's something like pathfinding, which isn't really AI because a path either works or it doesn't: there is very little interpretation for good and bad pathfinding. But most people do consider it AI. The goal there is: does it work, is it fast? And then everything else has to fall under that. Does it take the right amount of memory? Things like that.

For something like computer player AI which is where the computer player basically does the same things in the game that the human does. For an Ensemble game the goal there is: is it fun? The most primary goal isn't about whether or not it doesn't cheat, it isn't about whether or not it's good, it isn't about necessarily how much memory it takes, it's whether it's fun to play against.

Once we address that, or we are on the road to that, then we start worry about things that are attractive to the hardcore audience. Our AI's have never cheated except on the level that we say that they cheat and I think that our hardcore audience has responded well to that.

What do you mean, "on the level that we say that they cheat"?

We have five difficulty levels and hardest is kind of like the nightmare level in Doom. You know, it's not a level that normal people can play at. It's not a level that they're supposed to play at, it's there really just to see kind of how over the top could you go. The first four difficulty levels, easy through hard, the AI doesn't cheat and we've always been really happy that we've been able to make a fun AI that doesn't have to cheat to be fun. I think that when you sit down to play a game of Age of Empires and you play against the AI, it feels better playing against that because you know that he's playing a fair game. In other RTS games you know that as soon as the game starts up the computer player knows exactly where your town is. In Age, they have to explore just like you do to find your town. You know, you pick up your town and move it, they have to explore to find it again.

Is CP cheating lazy programming or is it the only realistic way for a computer to compete with a human player?

I wouldn't say that it's generally lazy programming. I'm sure in some cases that the AI cheats because people are lazy. More often, the people who I think kind of understand the role of AI in the game, they cheat because they think it makes it more fun. I think that everybody would write a non cheating AI if they could, if they had the time and the resources to do it. So cheating is a way where you get from a less hardcore user viewpoint, you know 80% of the bang for the buck with a lot less effort than building an AI that doesn't cheat. For a lot of people-myself included at least-once you know the AI cheats I pretty much don't want to play against it anymore because it's not a fair game. Whereas with Age AI, it's fun to play because you know it's fair. Cheating can be used to overcome some of the shortcomings that the computer player has. While a human player can always innovate and go read the latest strategy off the net, the AI has a hard time doing that. There hasn't really been affective learning AI in games at all at the level of complexity needed to do a strategy game.

Now why is that?

Strategy games are hard! You look at chess, which is always the preeminent example of good AI—or least the one that the general public understands. You know, Deep Blue and Kasparov and all that stuff like that. Chess by itself is a really simple game. You know, there's 16 or 17 different rules depending on how you break it down and there's more if you break it down a little bit farther but it's a pretty small set of rules. You take a game like Age though, there are significantly many more rules and we add in a random map complexity. It removes the ability really to prescript an opening strategy. A lot of Deep Blue's power is that computers are really good at remembering lots of different things and kind of figuring it out that this particular placement of pieces and this particular order of moves is this strategy and here's a good defense against this strategy. That has to be encapsulated in a strategy game, or at least the way that we've chosen to do it where it doesn't cheat, in a way that is reactive and still works within the game construct. If you see the human player running to your town with ten cavalry, it's certainly really easy to figure out a spot on the map that's nearby that he doesn't have the line of sight to, pop in 20 pikemen, and then go tell them to attack. Once the human player knows that, the game I think, is less fun. It's much harder (for the computer) to say, "Oh, he's attacking me with ten cavalry, do I have a barracks? Yes I do. Okay this barracks is too far away, so I'm going to build another one closer and then train the guys out of that one and then remember that when those pikemen are built that they need to go attack these cavalry." And then you get half way through training the 20 pikemen and the cavalry guys are gone. Suddenly you have ten pikemen with nothing to do. Then he (human) comes in with archers, which eat pikemen up. So how do you react to that situation? There's a lot more variables in a strategy game that make a lot of the traditional, academic approaches much less immediately applicable.

So you can't actually yet get the computer to react to what I do? It has to have a certain set number of strategies?

No, I wouldn't go… that's how it worked in Age I, which was kind of the first new game that we had done, the first game that I wrote the AI for. For a good RTS game there's always several counter strategies to any strategy that a player chooses to play. What we do with our scripting language is an AI designer will write some content that says, "if you are getting attacked by this particular type of strategy, with all this historical knowledge about the game, then you want to attack with one of these strategies and you want to pick them according to this metric where… You know, if you're getting attacked by Monks in Age II, and you're on a water map, and the Monks are close enough to the shore that Canon Galleys can hit them, and you have some Cannon Galleys, then you can just send them over. Else, you need to go through and create a whole landing party and things like that." It's the more mechanical part of the script that the designers write as well as a lot of the high level strategies. The game content for a Second Age British Archer Flush, you know a rush from the second age, the basic content of that is encapsulated in 6 or 7 different items - things that you're supposed to do. What the AI does, is it figures out how to make those happen as quickly as possible. The difference is that if we ship the game, three months later someone has developed a strategy that the AI just doesn't handle. To contrast that with Deep Blue, it's got pretty much every known chess strategy in it somewhere. It's just a question of can it find it fast enough to do the right thing.

Is it your programmers that come up with these strategies or your testers?

It's really more of our designers. We spend a lot of time on AI. Just to look at our current strategy game project which is our third one, RTS 3, which is a super creative code name (laughs). We have a full time designer that just writes AI code in our scripting language. So it's not really code in the sense of what a programmer writes.

It's a program within a program?

Yeah, it's a slimmed down version of C.

So anything the designer does can't affect the main engine code.

He tells the main engine what to do. In the same way that the human sits down, and moves the mouse, and selects guys, and right clicks to tell them to move places, the script that the AI designer writes causes the computer units to work in the same way. We have a lot new systems in the RTS 3 game where you can write AI at lots of different levels. You can leave a lot of the tactical execution of your strategy up to the computer player AI that the programmers write or you can micromanage it a lot more.

Do you ever put out a patch when someone does come up with one of these brilliant strategies that the computer can't handle?

We certainly do update the AI. We had to do a patch on Age II right after we shipped because there was a bug in the AI script code. If you played an easy game in deathmatch mode with 200 units per player, the AI would resign in 5 minutes. It was just a bug. The biggest change that we saw was, actually when we hired our current AI designer and had him come in, he came in after we shipped Age II, but before we shipped the Conqueror's expansion pack. He ended up rewriting a lot of the script code and ended up improving the computer player's Castle time by 4 or 5 minutes pretty much across the board and as much as 6 or 7 minutes in extreme cases. And it was a lot of just optimization. It's always an iterative thing and he certainly employed a lot of different, newer strategies than we originally employed in Age II. For Age II, we have a delivery mechanism for additional random map scripts and we issue a new ones every 2 or 3 weeks. You download them and the game understands that and it works well. We don't have a delivery mechanism in Age II where we can do new AI scripts other than with Conqueror's which kind of put them in. For RTS 3 we certainly are going to do that and there will be a lot of personality type AI's that people can write, put up on web sites and download.

Are the personality AI's going to have a completely different style of play based on who creates it?

That's certainly the goal. From a philosophical standpoint, in my opinion, any one person only knows how to write a single really good AI. If you sat down to write an AI, you're going to write one that basically mimics your play style that plays as well as you can make it play in your play style. If you try to make it play a different way, it's going to be suboptimal because you don't understand the nuances of that play style as well because that's not the way you play. People who really, truly can master more than one play style… Like, we have a guy on staff, Kevin Holme, "The Sheriff," who is usually ranked the number one Age player in the world, he really, truly can play different strategies. I've never seen anyone else who can do that. I can play a different strategy than I normally play but I find that I'm not very good at it.

When programming strategies, do you hand them out to a team saying, "you do one strategy, you do one, you do another, etc.?"

That's one of the things we're talking about. Mike Kidd, he can play different strategies pretty well and so we're hoping to get several different types of play out of him. If I write one, it will be different from his. If Kevin writes, hopefully 6 or 20 or whatever he can write, we're going to get different ones out of him. We'll have some of our less experienced players, and they'll probably try to write some too, and those would be some of the easier personalities. That's another thing, it's hard for somebody who's really good at the game to do the right things to dumb down their AI to play a fun game at an easier difficulty level.

How do differentiate between complexity of AI and what makes it fun?

It's a hard thing to do. Are you familiar with Maslow's Hierarchy of Needs? It's basically a pyramid of basic human needs. First you need to worry about food and shelter, then you can worry about respect or something. At the top is self-actualization in which you understand human meaning and you're happy with yourself. There's no point in worrying about that if you don't have food and shelter because in 2 weeks you'll die. Kind of equate a lot of the complexity versus fun tradeoffs to that. You need to get the computer player to play the game at all, and then he has to play the game well. And then, once he plays the game well, and once you have a good computer player AI who does or doesn't cheat according to whatever you decide to do for the project, who can stand toe to toe with the best people on the game, then you can worry about making it fun. If you worry about making it fun before then, you're going to have an AI that might be fun for a good segment of the market, certainly the more numerous ones, but the hardcore people may never get the AI that competes with them.

To make the computer player AI fun you generally have to sometimes take less advantage of things that he does well. A good example is in Age II, the computer player is very good at micromanagement because he has a lot of CPU cycles and he remembers everything perfectly. A lot of people find it very un-fun to play against computer player archers on a really hard difficulty level because he backs them up all the time. He shoots at you, he backs them up just enough to where they can shoot again, he shoots at you immediately, he back them up again… And it's really hard to play against him that way. That's one of things that he leverages in order to work against the human player's ability to innovate and to exploit the things that the computer player doesn't do well. A lot of human players figured out that walls were a very difficult thing for RTS AI to handle. Age handles them as well as I've seen anything do it but if you build 6 or 7 layers of walls around your town, the computer player pretty much isn't going to get in. And you can win every game if you do that. We haven't got to the point where the computer player is good enough at breaking down walls where we can even figure out how to worry about making you have fun when he does that.

We know from the archer case that for a lot of people, when the archers back up, that crosses the line of the computer player exploiting something that he is good at, that makes him play better, that's still fair, but it's not fun. But we still employ that at the higher difficulty levels because we want the game to be hard. For people who want to play against a hard computer player, the computer player needs to employ those mechanisms because otherwise you don't as hard a game as possible. Now we don't cross the line to cheating because that's not what we want to do.

At the easier difficulty levels though, when the computer player doesn't need the advantage provided by archers that back up, we don't do that because it's more fun if we don't.

So how do you gauge whether that's fun or not?

We have a unique company at Ensemble anyway, you know we've got 65 people who are passionate about doing great computer games. We require everybody to playtest once a week. We have a scheduled playtest time. You come in at 10 o'clock in the morning, you playtest for a couple of hours, we actually bring lunch in for those people so they don't have to worry about going to lunch. You playtest the game, the designers are out there and the QA people are out there watching you play, and then you turn around and give them your feedback. You say, "well this sucked, and I didn't like this, I did like this, why isn't more stuff like that?" So when we playtest against the AI we get a lot of the feedback that way. We depend I think on our company to keep us going down the right path. We have a lot of different skill levels in the company from super-ultra-hardcore with "The Sheriff" to super-non-hardcore with our HR manager or reception. If they love games and they play, their feedback is different.

As we get into the Beta cycles for the game, we get a lot of statistical feedback. But the bulk of the early portion of the game is decided by people at Ensemble playing the game. Once we get to the point where we've made a lot of the decisions, we have enough of the game running to do Betas, we run a few publisher tests and stuff, then we start running through the archer example. We didn't actually do this, but if we wanted to we could say, "Okay, we ran it through a thousand people of intermediate skill level and 66.2% of the subject hate that feature, so we have enough to justify the statistics, so we should probably take it out." It's not an exact science and I think until computer player AI gets good enough to where, across the board you can actually have him play less optimally. We'll get there eventually. You pick and choose the areas where he does play well or at least more so than some other areas. That's kind of like the pyramid thing. That's when you can worry about the fun aspect.

I've heard a lot of people say, "Once you've played one RTS you've played them all!" What are you doing to prove that wrong?

I would never agree with that statement. Age is different than Starcraft or Warcraft and both of those are different than Command and Conquer. C&C and Red Alert and I would put maybe even Total Annihilation in that category too. They're fast games, they're not focused much on resource gathering, very little on exploration, in fact you start in those games with the map generally explored. Contrasting that to Age, Age is more classical in terms of RTS games. It's about exploration, it's about having a random map because that gives you the replayability, it's about spending a lot of time building up your town and resource gathering. Whereas some people don't want to do resource gathering, that's a core component of it. We have deathmatch which gives you a billion of every resource so you don't have to do that and that's an option that some people do. We never play that in the office because we hate it. It's not what the core game is about. The core game is about interesting decisions. You start off the game, you've got the choice, you've got a couple of guys, you can chop some wood, you can build some houses and as you go down the path it's a inverse pyramid in terms of decisions and making strategic choices. Age games take longer than most other strategy games because they have, I think, more decisions to make and they're larger games in scope. They have a lot of different civilizations. They don't seem as differentiated when as Starcraft civilizations when you sit down and play, but in the end there is a lot of depth there.

From an AI standpoint, I'm biased but I think our AI has always been, if not the best, then certainly Age II is the best. The random map thing, I think, is what clearly sets us apart. Scenario AI, you're playing AI that's really more triggers. You can fake a lot of things by using a good trigger system.

Is there any way that you could affect player choice so that they can have an effect on the Campaign story lines?

The simple answer to that is, "yes of course." It actually starts with the random map AI and then we reuse that for the scenarios. It's in the scenario designers control to let the AI play more randomly or less randomly. They usually choose, because they are telling a story and the best stories tend to be linear. Despite the kind of cool factor that as a hardcore gamer I love that I can influence the story, the type of games you play where you can influence the story generally tend to have less powerful stories. We're aiming at a big, broad mass market who, in reality, seem to care very little about non-linear stories. As we get more advanced I think we have options to change that. As the art form goes through maturation we can figure out, and this is certainly a focus for RTS 3, now that we've done more campaigns from Age I to Age II and the Conqueror's we feel really happy with where our campaign quality is from a very linear standpoint. Actually one of the interesting things about RTS 3 is that we're trying to introduce some non-linearity to the story.

The flip side to that is that the campaigns in Age have always been a part of the experience. We are always happy to say that we put a lot of things in the box. The random map is where we've always hung our replayability hat both for multiplayer and for single player. We're talking about random campaigns for RTS 3 to help take that feature to the next level. I can't say too much more about that… But it's a hard problem you know? In an area with a linear mechanism, how do you take a story and generate it randomly so that it's any fun to play? You know, it's not where 95% of the story is exactly the same but instead of capturing the Sword of Fubar, you're capturing the Cup of Fubar. We wouldn't do that because that's not a very quality implementation. The goal for RTS 3 is to be the best single player experience ever in any game on any platform. That's a very high goal to achieve and I think that we can get close if not achieve it.