Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
November 1, 2014
arrowPress Releases
November 1, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

Opinion: The Spector of game AI
Opinion: The Spector of game AI
August 28, 2012 | By Luke Dicken

August 28, 2012 | By Luke Dicken
More: Console/PC, Social/Online, Smartphone/Tablet, Indie, Design

In this reprinted #altdevblogaday opinion piece, Strathclyde Artificial Intelligence and Games researcher Luke Dicken riffs on Warren Spector's recent AI comments, and explains what AI can do for your games.

Warren Spector recently gave an interview to Eurogamer that's circulating pretty widely now about the need for better AI in games. Specifically what he'd like to see is people like John Carmack and Tim Sweeney stop focusing on graphics and start working on creating believable characters and immersive worlds.

Of course, I've got some issues with this. These guys are graphics programmers, they aren't AI experts. Thinking that a graphics guy can just up sticks and refocus on AI is effectively part of the cultural issue AI has – it isn't considered a serious discipline, it isn't "hard," it isn't something people are experts in already, so let's call in some graphics programmers to solve the issue!

Of course, this is completely untrue and shows a bit of contempt for the AI that does exist and the experts who are working on it right now. We don't need John Carmack to solve AI for us, we need people to listen to the Carmack-esque characters who already exist in AI and who are ignored because "players don't want good AI".

However, this minor piece of bitterness aside, what Warren is talking about makes a lot of sense, and it's something I've been saying a lot over the past couple of months as part of a lecture tour that's taken me to South America, Europe and across the USA.

One of the key themes I've been talking about is AI outreach and advocacy – helping to explain what AI is and how we can use it to make better games. So, because Warren has set the stage so well by bringing it up, here is the article version of that presentation!

Why so scared?

AI is seen as a really scary topic by a lot of people. I think a big part of it is that we've been culturally conditioned to see intelligent machines as an inherently bad thing. We've had decades of books and movies dedicated to the enslavement of humanity, if not an outright apocalypse, all because of robots and AI systems getting a little too smart for our own good. It makes people kind of wary. But it doesn't have to be that way!

All AI really boils down to is the applied science of decisions. There are whole branches of maths dedicated to analysis of decisions, Decision Theory being an obvious one, but also Game Theory and even Economics. These all allow us to theorize about how we should react in a given situation, and after we do all the algorithmic analysis, we end up with a neat result on a whiteboard and we walk away – problem solved. AI is about putting that solution into practice, whether it be in some simulated environment or with a robot.

Why is AI scary? Well, this guy has been our spokesman for nearly 20 years…
But the big question is, where do we see these kinds of decisions? Typically when gamers talk about AI in the context of games you might hear that "The AI did such-and-such," and what they really mean is the enemy in the game.

There are an awful lot of decisions happening there, it takes a surprising amount of sophistication to create characters that can believably run into the player's field of view and convincingly fall over when killed. And whilst this is what a lot of Game AI focuses on as a discipline – what you might call opponent AI – there's a lot more possible.

People often say that players don't want better AI in games, this is what they're talking about. And to an extent they're right, because what they mean is that players don't want opponents that can out-think them.

Part of my advocacy has been to address this specifically, especially in the academic community where "better" typically means stronger and more capable. If that was truly the goal of Game AI, as Blizzard's Schwab pointed out at GDC '11, we could just make NPCs that were faster, stronger and had more powerful attacks.

Making a game more challenging in and of itself isn't hard. Academics – especially AI academics – like to talk in terms of optimal solutions, so one of the breakthroughs I've had is to explain Game AI as AI that tries to optimize "fun". Some people get it, others then want to pontificate about an algorithmic definition of fun, but that doesn't alter that as a core concept, this is what we are trying to achieve by using AI in games.

What I want to do, though, is explain to people how we can use AI to do this. I'm not going to deal in technical details, because what I want to do is explain this in a way that designers and producers can follow, and implementations already exist out there.

The best game ever made

I want to start out by asserting that Dungeons & Dragons could quite easily be described as the best game ever made. It, and the table-top genre as a whole, has endured in a way no video game has to date. And its supporters aren't growing tired of it; it's just a bit of a pain to organize compared to firing up World of Warcraft.

The great thing about D&D is that it's not a game itself; it's a rules framework in which games can be created and players aren't restricted in what they can do. Of course that means that players will try to do weird and wonderful things, things that could never be predicted beforehand, and that's where D&D can come into its own.

The thing that makes this possible is the human Dungeon Master who controls the game. The Dungeon Master is part of the game, and part of the gamer group. Before the game starts, they create the world and build up an idea of what the plot of the game will be.

It's important to note that since the players drive a lot of the game, the DM can't plan out the entire storyline in advance, because as any good strategist knows, "No battle plan survives first contact with the enemy," and players need to be able to shape the story that is unfolding in their own way.

Invariably however, this unfolding process will lead to a combat encounter, which requires a map to be created and populated with creatures for the players to battle. There may be plot devices as part of the encounter or specific elements that foreshadow something to come later which combine to create a subtly cohesive experience for the player.

Once the encounter is created and the players are engaged in it, the DM will take responsibility for controlling whatever creatures have been placed into the map. This is a fairly interesting part of the role, because the DM has to balance the need to take believable actions that would make sense for those creatures against the need to make an exciting – and not too challenging – experience for the players.

If the players will die from one more hit from a goblin archer, it doesn't make sense for the archer to turn and fire into the wall. This isn't a believable action though, and players in general won't like that there is only an illusion of risk.

Schwab described this as "Losing with style," and it's a key aspect to this kind of player experience management – we don't want to be blatantly throwing the game in favour of the player, but at the same time, we don't want to be insensitive to the fact that they are playing a game and don't want to be frustrated or have the game end prematurely.

One of the most important ways to manage this is to have a good understanding of who you are running the game for, knowing what types of player are in the party and how they will react in certain situations. This allows the DM to have a much better handle on how to stage manage the game to get the desired outcome, as well as tailoring the content to suit particular player types – if you know that players in your group tend to favor finding diplomatic solutions rather than confrontation, you can ensure that the options for this kind of play are available.

Equally, if you have a player who is liable to charge at every band of enemies presented, you might like to prepare "surprises" for that player to exploit that they're going to do something foolhardy.

The Dungeon Master is, in my opinion, a significant part of what makes D&D and games like it so successful. They have a longevity completely unmatched by any other games from their era, and continue to enjoy popularity and a zealous fan-base.

To my mind, they represent the "Gold Standard" of what we should be trying to emulate when we make games today – not necessarily in terms of setting and tone, but certainly the design philosophy of empowering players to make choices and have real, tangible impact on the worlds we are playing in.

Of course, the obvious way to do this would be to ship a DM with every game, but I've been assured that cramming guys into retail boxes specifically to look over the player's shoulder and tweak the game in real time would be considered a human rights violation. … Hippies.

So instead, what we typically do is pre-bake a lot of the DM's job during development. We fix the whole plot of the game upfront and reduce it to a sequence of combat encounters stitched together with cutscenes. We play those encounters out in a set of fixed levels, crafted by a designer once and then never changed, no matter how many times a player works through them.

And then, we spend the vast majority of our time focusing on controlling the minions within the encounters – to such an extent that this almost becomes the totality of the wider perception of what Game AI is.

Of course, it is so much more than that so I want to close out by discussing briefly how AI can be used to start to replicate the DM's role, and hopefully create powerful, engaging experiences as a result.

AI for storytelling

This one is a bit over my head, but we can use AI to dynamically tell interesting stories, to take into account the player's actions and adapt the story to take this into consideration. We've started to see this becoming increasingly mainstream lately, with BioWare being one of the big pioneers of some of the lowest-hanging fruit here with branching narrative.

This is more of a '70s era expert system than what we think of as true AI, but consider a game such as Mass Effect, where your choices at one part of the game are going to influence what happens and what choices are offered later. At a basic level, you might think of this as a choose your own adventure book, but even that rapidly becomes pretty complex.

Now add the nuances that Mass Effect and the like introduce into dialogue, recurring characters and so on, and the combinatorial explosion on the potential state graph is pretty intense. We can go a stage further though, because not only can we represent an enumeration of potential states using AI techniques to effectively manage this, but we can start to introduce actions within the world that are the AI system steering the player towards certain states in this notional graph.

If we specifically want the player to arrive at a particular point of the story, we can push them back in that direction and begin to constrain their choices. If done carefully, we can give the illusion of complete choice, whilst still stage managing the experience. We can also build in concepts from literature to this, and manage the dramatic pacing of the story by modifying how components are shown to the player.

Choosing your own path through an entire trilogy proved to be so engaging to players, that they nearly revolted when this choice was removed.

There's a strong and growing Interactive Fiction movement that has some great resources on this subject. A great place to start if you are able is "Beyond Eliza: Constructing Socially Engaging AI" from this year's GDC. For those without Vault access, Leigh Alexander wrote a summary of the session on Gamasutra that is a good substitute.

For those wanting more detail, you could do a lot worse than check out University of Teeside's Marc Cavazza, who has a lot of publications centered on the theme.

AI for encounter design

Procedural Content Generation is one of the latest crazes sweeping through the game development community. At its most basic, it says that creating content, whether that be quests or a varied array of weapons or even levels for the game, is a time-consuming process.

For a certain amount of overhead, we can instead create an AI process that will make that content for us, and make as much of it as we need. Skyrim demonstrated this effectively with the Radiant quest system, which enabled the developers to create a seemingly limitless range of side-missions without ever having to write one of them by hand. The system could pick, in a mix-and-match style, a location for the quest, a task, a target and then assign it to the player.

Tiny Wings is another great example, where every day the levels the player experiences change – without a way of generating these levels algorithmically, it wouldn't be feasible to do this since it would be so time-consuming.

No discussion of PCG is complete without a Tiny Wings reference!
At its heart, procedural content is about creating things dynamically, and may seem to be random, but rarely is. Randomness in general is a bad idea when you are talking about replacing a talented content creator such as a level designer – there's a reason these folks get paid to make content, and it's not because they can be replaced by glorified dice.

With that said, it's often possible to capture something of their insight and build that into the generation system, to create the kinds of content that they would. Using Tiny Wings again, each level has a different parametric description, so the overall feel of the level is kept the same and the difficulty progression varies in the same way each time, despite the specific details changing day-to-day.

Procedural generation is something that's really close to my heart right now as myself and Heather Decker-Davis have been relying on it heavily for a casual title we've been working on, and in turn it has shaped a lot of our workflow and tools.

You can hear more about our experiences and some tips we shared for working with procedural methods in this presentation, which we gave at the No Show Conference in July. You might also find this article by Notch discussing terrain generation in Minecraft of interest, and again if you have GDC Vault access, the guys behind Spelunky gave a great overview of their approach last year.

AI for character control

As I said earlier, this is traditionally an area that gets an awful lot of attention, so I'm going to gloss over it a little bit, except to say that there are a broad range of techniques that you can use to create characters, such as Behavior Trees, Goal-Oriented Action Planners, Hierarchical Task Networks and even Finite State Machines.

Ultimately, though, you need to understand what you are trying to achieve – is it to provide the player with a challenging opponent, or an engaging companion? The AI techniques required to die convincingly are separate from those required to create a believable set of responses to the weird and wonderful things that the player might decide to do.

If you intend to make opponents, you need to consider if they work in squads, and whether a squad is a loose collection of affiliated enemies, or a well-trained group following specific orders, perhaps from an officer within the group. The Halo franchise demonstrated that small squads with officers could implement a morale system effectively and have squad cohesion disintegrate with the death of the officer.

These design choices are going to have a direct impact on what AI techniques are going to be most appropriate to create the desired feel to the characters in our game worlds, and it can be some of the most immersion breaking moments for players when the characters don't act the way they expect.

They are willing to believe that an alien swarm has taken over New York and the government has ordered the military to kill everyone involved, but they aren't willing to believe that it's possible to kill one of a pair of soldiers without having the other react in some way to his companion's death.

There are lots of resources for reading more about this kind of AI, from through to a large portion of the AI Summit held at GDC, available through the Vault. The AI Game Programming Wisdom series of textbooks is also a good source of material.

And when it comes to understanding the rationale behind the decisions characters might make in specific circumstances, I also highly recommend "Predictably Irrational" by Dan Ariely, which is a book about so-called "Behavioral Economics," or where decisions meet psychology head on.

AI For player modeling

Player modeling in games is a relatively new approach that has yet to see wide adoption, but which has the potential to be a bit of a game changer, especially when combined with some of the other techniques already discussed. At its most basic, player modeling is something we're already pretty familiar with whenever we play a game with humans; we try at some level to get a sense for who they are and how they play.

This is most often seen in games like Poker, where the stereotypical classifications of players, such as "Tight" and "Loose" have become part of our vernacular. In these competitive games, we use this kind of classification to predict an opponent's behavior and exploit this, but again this isn't really the aim of Game AI – sure it's great when an enemy opponent adapts to your tactics and challenges you to try harder, but we can use this technique for so much more than this.

Silent Hill: Shattered Memories is a great example of player modeling in action. In the opening scene of the game, you find yourself in a doctor's office undergoing a battery of psychological tests. Unknown to the player, however, the scene is set up in such a way that although the character is being given these tests, really the player's responses to them is what matters.

In truth, the testing is being done on the player, and it's actually a standard psychological profile being built up. Based on this, the player is put into a category, and throughout the game, different content is shown to them dependent on their category. It doesn't change the overall story of the game, but it does change the tone and feel of the experience.

This is the same scene, as experienced by two different types of player

But what happens when you don't have the luxury of running a complete personality test on the player as the game begins? Let's face it, games would rapidly begin to feel pretty dull if they all had to find a justification for doing this! It turns out that just as in Poker, we can start to classify players as we observe their playstyle, we can do the same thing with other games.

In 2009, Drachen et al at ITU Copenhagen were able to analyze data from playthroughs of the AAA title Tomb Raider: Underworld. What they found was that by applying some machine learning approaches, they could extract a number of stereotypes of players that people would broadly fall into, namely "Veterans," "Solvers," "Pacifists," and "Runners." This was based entirely on in-game analytics, the kind of data we all have access to already.

Using AI systems then, we can learn a lot more about our players and how they interact with the game. But we can also feed that back into our content generation or narrative systems to start customizing the experience directly to specific groups of individuals. This is something we're seeing explored in the academic world through such things as the Mario Level Generation competition that has been running for the last couple of years.

It's something that we're starting to see come to "real" games a bit, and in particular it's something that myself and Heather have been working to put into our indie title. For us, it's pretty much a no-brainer. Our game is based around a simple maze system, and this is broadly what our PCG system creates.

If we can detect players who are interested in exploring the maze and ensuring they have collected everything before progressing, then we can adapt our content to prefer to give them more sprawling mazes with a wider range of things to collect. Equally, if we find players are more liable to dart straight to the maze exit and take more risks with the hazards in the maze, then we adapt to suit those tastes too and provide a more action-oriented experience.

As with the human DM, understanding the players tastes and desires means that we can cater more directly to specific individuals and get better engagement with them. That means they have a better experience, more fun and means we're making better games!

The end

I really hope that this post has helped you to understand a little bit more what AI can do for your games – in a very real way, Warren Spector is bang on the money, AI has the potential to be the next big thing in game development but we need to get more designers who aren't intimidated by it!

If you want to learn more, there are some great resources online that can help you get started, and of course every year the AI Game Programmers Guild hosts the AI Summit at GDC which is a great source of more inspiration. And who knows, maybe you'll run into Warren there!

[This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]

Related Jobs

Rensselaer Polytechnic Institute
Rensselaer Polytechnic Institute — Troy, New York, United States

Assistant Professor in Music and Media
The College of New Jersey
The College of New Jersey — Ewing, New Jersey, United States

Assistant Professor - Interactive Multi Media - Tenure Track
Next Games
Next Games — Helsinki, Finland

Senior Level Designer
Magic Leap, Inc.
Magic Leap, Inc. — Wellington, New Zealand

Level Designer


Jacob Germany
profile image
A few issue with some of the assumptions in the preface:

1) Without speaking to John Carmack and Tim Sweeney's talents or skills, any field (or entire industry, for that matter) will always be improved by adding new perspectives from outside traditional field perspectives. And that any subset of development is "too hard" for talented developers simply because they've chosen a different preferred expertise is a bit odd and, I think, overestimates the difficulty of any particular field.

2) I couldn't tell if the author was entirely serious, but I don't think anyone considers AI "scary" because of science fiction. It's more of a broad cultural edifice in gaming where AI focuses on combat because *everything* focuses on combat. The entirety of design (from UI to rules to AI) places more focus on combat than, say, relationships or communication or emotional manipulation of the player.

3) "[players] mean is that players don't want opponents that can out-think them. Part of my advocacy has been to address this specifically, especially in the academic community where 'better' typically means stronger and more capable. If that was truly the goal of Game AI, we could just make NPCs that were faster, stronger and had more powerful attacks."

I've always been underwhelmed by this argument. If players really wanted enemies to have a definitive limit in intelligence and skill at playing a specific game ruleset, multiplayer wouldn't nearly be as popular as it is. While AI can certainly surpass the possible skill of even the greatest players in the case of extremely limited rulesets (e.g. chess), sufficiently complicated spaces will always be beaten by players. I think, then, that (at least some) players would appreciate enemy AI becoming more strategic and 'intelligent'.

Danny Bernal
profile image
People want smarter AI. what they dont want is AI that cheats.

AI that never misses a shot is cheating.
AI that never fails to directly find you is cheating.
AI that avoids all attacks flawlessly is cheating.

AI that reacts to proximity/vision/audio/etc is smart.
AI that actually searches for players in likely places and "learns" of their location through process is smart.
AI that reacts or attempts to react to an attack based on the same conditions a player might have.

I think we get the point haha. I think what people really want is personality and a fair chance to compete with AI. Difficulty is not the issue. As long as players can see that an opportunity obviously exists somewhere then they will enjoy trying to win, even when stakes are high.

Jacob Germany
profile image
I agree! I dislike it when people say that AI shouldn't be "too smart" when that was never the problem. The problem was that more and more intelligent AI was applied to bots that still have the quirks of the days of yore when we "cheated" to create difficulty because AI was so dismal. When we had to create difficulty by having NPCs find you from anywhere on the map, because a more realistic set of senses was too complicated. When they all could hit you, in a split second, and never fail. Give *those* guys more intelligence and, yeah, players will be irritated.

But create a smart AI that has player-like limitations and there really isn't a reasonable limit.

Winter D
profile image
Speaking as a player:

I want AI that is an equal opponent...WITHOUT it relying on its advanced knowledge of the rule system. I don't want to be beaten because I didn't realize that unit X could do Y better if I gave it upgrade Z. I want to be beaten because the AI tricked my army into following it's army into an ambush. I want to feel like it outsmarted me, not like it consulted the rulebook and used the optimal strategy for a certain situation.

Mike Scott
profile image
@Danny - The problem is that it's difficult to create a smart AI that that doesn't appear to cheat. I agree with the concept of limiting an enemy's awareness to it's surroundings (and the player location, for example) at the same level the player experiences it (with a level of additional "fog" in the detection system to act as a difficulty level). However, if you do something like have the AI "learn" from it's interactions with the player, such as determining where a player is likely to be, and then produce algorithms based on countering that info, the Enemy will quickly seem to "automatically know" where a player is. The player's perception will be that the enemy is cheating.

We've all seen/heard on multiplayer sessions where someone complains that another live opponent is cheating, merely because of a skill level comparison, or knowledge of a level, or specific weapons, or simply using a gameplay style that proves to be an effective counter to what that player is doing, etc. All of which an AI is in-danger of falling into if it's given even an "adequate" problem solving algorithm. Granted, the player might be a whiny, ineffectual player, berating an opponent just to justify their own inadequacies, but that's the nature of the beast in creating a game... some of the players will not be experts, and may need some coddling if they are to maintain interest in the game.

You want the player to believe your enemies are making choices/decisions based on observations made with senses that match the player's abilities. That's not the same thing as actually creating a system to do just that.

Gameplay at it's core is giving the players the skills and ability to succeed, while providing the perception of difficulty, a reward system for using those skills, and possible failure, while maintaining a certain level of fun.

It's something to keep in-mind when planning AI.

Jacob Germany
profile image
@Mike I think Danny meant is an NPC that learns of a player's location by looking in "hiding spots" or "nooks and crannies", not by using a genetic algorithm to create a fuzzy graph of the level's map. But, I could be wrong, since I'm not Danny.

Kevin Patterson
profile image
Games like Seaman for the Dreamcast are interesting, and would seem to be great for showcasing AI, yet we haven't had any games like that this generation (I think there was one in japan..maybe).
Milo's demo looked to be the next step, but that never came to fruition, which is a shame.

I would think having a console with a built in AI character would be a pretty neat feature, where it could remind you of events and holidays, ask questions, and learn about you over time. With Kinect's voice and motion ability, it would seem like a natural next step, but nothing has happened like that this gen, maybe next?

Joshua Darlington
profile image
Thanks for the Cavazza link! Fully geeking out on that.

profile image
There is a lot of work to be done in this area, and it doesn't help that the academic world is not very much further ahead that the entertainment sector. 3D graphics had a large body of work related to plane transformations and linear algebra to draw upon, no such body exists for AI (academic AI research is generally concerned with automaton of tasks such as medical diagnostics, or the classic "cookie shape sensor").

I guess the real problem is one of risk management. If your shaders don't work and the game looks kinda goofy, the game is still playable. If you mess up the AI such that it's over the top or useless, the game is much more heavily impacted. A nice demonstration of this is Jeff Lait's "Smart Kobold" - which shows that good AI is maybe not always a desireable thing in games.

Brad Borne
profile image
I heard most of his thoughts on this from the Game Informer Replay of Deus Ex that he participated in, but it sounded to me like a 'what if' thought about brilliant minds and brute force manpower that goes into graphics nowadays.

I don't know, listening to how John Carmack solved crazy problems to get 3d on early hardware, and how brilliant he is, I'm not sure that he wouldn't be able to get NPCs behaving in ways that better their games, if he really put his mind to it.

He's a graphics programmer, but that certainly didn't stop him from improving VR hardware.

And he was specifically talking about NPC behavior, not procedural content or dynamic difficulty or whatever.