Poker AI
When I programmed the AI for Left Field's World Series of Poker, the
AI computation was basically the same for each difficultly level.
The
computer would calculate the odds of winning based on the known cards,
and an estimate of the opponent's hand strength based on betting
history. The odds would then be used to calculate a rate of return,
which would be used to decide if they would fold, call, or raise.
There were many special case rules and exceptions, but that's the
basics. The AI players would all make the same extensive computations,
running tens of thousands of simulated hands through an evaluator to
calculate the rate of return.
After these calculations were performed,
only then would the differentiation be performed. At that point, the
best players would play their best move, and the weak AI players would
make intelligent mistakes.
For weak poker AI, an intelligent mistake consists of figuring out
what you should do, and then not doing it, so long as not doing it does
not make you look stupid.
For example, if the human player just put in
a big raise, yet you know there's a 75 percent chance your hand is the
best, then an intelligent mistake would be to fold. The odds are the AI
would win, yet we are simulating a weak human player, and weak human
players often fold to a large raise when they are unclear on their
odds.
Conversely, weak human players often call when their chances are
weak. It's a natural thing to do and allows us to reduce the strength
of the AI player, without it looking artificially stupid.
 These intelligent mistakes were implemented in a probabilistic
manner. The fake-stupid AI would not always fold when the human player
seemed to be bluffing -- it was just more likely to.
This worked very well
in the highly random game of poker, because the player could never tell
in any individual situation if the AI was actually making a mistake.
Since the AI was still performing its full set of millions of
calculations, it never made mistakes that were inhumanly stupid, but
the layer of artificial stupidity brought on by increased recklessness
was enough to even the playing field and give the weak and average
human players an enjoyable game.
Artificial Inaccuracy
In pool and in shooters, the computer AI is blessed with an
omniscient accuracy. The shooter AI knows down to the billionth of an
inch exactly where you are, and could shoot your hat off your head from
five miles away. Similarly in pool, the AI knows the position of every
ball and can calculate where every ball will end up before it takes a
shot.
When I implemented my snooker AI, it could perfectly pot any ball
off two cushions, and would almost always get a perfect break of 147
every time it played (except when it potted the white due to its lack
of positional play).
It was obviously not a fun opponent to play
against, so even at the highest levels, the accuracy had to be reduced,
and the cushion shots had to be restricted to getting out of snookers.
Simply reducing the accuracy of the AI is not always the best way to
improve gameplay. As I found with the "positional play" in snooker,
random outcomes that happen to favor the computer are perceived as
being intentional. If the ball ends up in a good place, or the poker AI
makes a lucky call and wins on the river, it can be perceived as unfair
or even cheating.
So instead of reducing the accuracy, I'd suggest, as in chess, we
increase the accuracy. In order to provide an exciting and dynamic
game, the AI needs to manipulate the gameplay to create situations that
the player can exploit.
In pool this could mean, instead of blindly
taking a shot and not caring where the cue ball ends up, the AI should
deliberately fail to pot the ball and ensure that the cue ball ends up
in a place where the player can make a good shot.
In a shooter, the
enemy aliens should not simply randomly break from cover -- they should
sometimes break from cover when the player is close to them and panning
toward them. They should "accidentally" throw themselves into the line
of fire to make the game more interesting.
Luck of the Draw
Playing against a perfect opponent is no fun. But playing against a
crippled opponent is no fun either. To create more interesting
gameplay, we have to introduce the concepts of artificial stupidity and
intelligent mistakes.
Intelligent mistakes seem like failings on the
part of the AI, but are actually carefully calculated ways of throwing
the game that make it more entertaining for the player. This does not
remove the challenge, as the player still has to have a certain level
of skill.
For the programmer, adding intelligent mistakes is much more complex
than simply reducing the accuracy of the AI, but provides a much more
rewarding experience for the player.
Resources
Liden, Lars. "Artificial Stupidity: The Art of Intentional
Mistakes," in AI Game Programming Wisdom 2, Charles River Media, 2004.
http://lars.liden.cc/Publications/Downloads/2003_AIWisdom.pdf
Lopez, Steven. "Intelligent Mistakes," Chessbase News, 2005. http://www.chessbase.com/newsdetail.asp?newsid=2579
|
Printer-Friendly Version
When we were working on Rumble Racing, the goal was to make the AI opponents feel like people. We had a matrix of possibility for introduced errors, where at each pathing junction the AI had a choice to take a fast line or slow line. This matched up how people drove the tracks, where small errors in braking and taking the corners made the cars seem to go slower because of poor driving, not simply going slower.
Hey Chris - Rumble Racing was (and is, as I still play it today) one of those great games where one really got to see the AI do some things you'd expect a live opponent to pull off, so it was always a blast to race the same tracks multiple times and almost never see the same result.
g.
This gives moments of hesitation and panic which look extremely realistic (at least in something as simple as driving or fleeing). I've tried to think of a good way to cause the AI to flipflop between decisions only when they are in a panic, however that seems to require breaking good decision making skills (requiring a set amount of time before they can change action).
It's nice to see an article on this subject. A good deal of thought has to go into creating lifelike artificial behavior, and I'm glad to see people in the industry upping their game.
AI should have the same interface to game as the player. All positions and values that bot receives should be slightly inaccurate. AI algorithms should be developed separately from gameplay mechanisms.
If player detects that AI is throwing the game, immersion in gameworld suffers, as the player realises that the whole gameworld revolves around him. I believe the way to go is to introduce quirks and imperfections in AI behaviour, but not have it dominate the game by deciding at which point should the player start winning.
eg: I try to rush into the range and view of my opponents from whence they previously had no knowledge of my presence. Current A.I. would allow all notified opponents to 'snap to' my character and begin firing/ducking for cover with their speed and accuracy determined by difficulty. What I want to happen is for the opponents "psychology" to determine their actions, even though they may be 'trained soldiers.' I just initiated a "surprise attack" and expect them to act surprised depending on a combination of how alert the opponent may be and how efficiently I got within range.
It may sound like what I'm asking for is scripted A.I. set-pieces. In one way yes, but this has much larger scope. To me, realistic 'mistake making' A.I. mean an opponent that "feels." Will retreat because he fears, will fight because he's confident. I'm an artist so I don't know how to implement it, but it would be good to see the A.I. literally go "whoops"
Personally, I like gmaes that use the old D&D system of % to hit; etc; as I always feel like it's an even playing field between the player and the computer. Many games do a great job with AI (like Fallout 3 most recently), but I think some FPS and action games where the AI seems to be on a straight path to the protagonist can benefit from some rpg-style stats under the hood.
John Smith: Try explaining to your producer that you need to develop two different physics systems - one for the game to use, and one for the A.I. ;) Seriously though, your example assumes that with the right interfaces, an A.I. "brain" would act similarly to a human brain, which simply isn't true - even if your A.I. was unable to be accurate, it doesn't mean that it would feel like you're playing against anything analogous to a human opponent.
Games are experiences in which you need to create a suspension of belief, and the A.I. is a tool in that - there will never be a fair fight between an A.I. and a human, but with your A.I. making the right mistakes, you can trick your player into thinking that there is. And at the end of the day, that's all that matters! ;)
C&C 3 is case in point for this.
You mention the prospect of setting up a situation that the player can exploit. That's definitely a fine solution to the handicapping problem! My interest is in the preconditions for setting up such a situation.
As you show with the poker examples, there's usually a reason that a player makes an error, or a type of error which is more typical. (Folding to a big raise, against the odds. That one is a very good and clear example! Folding when "unclear" about the odds... I think this means that you can teach the machine which hands are complicated for a human to calculate the odds for.)
Let me give another example to sharpen the angle I'm getting at.
Let's say we were writing a chess program aimed at beginners (defining 'beginner' as not absolute novice, but rated under 1400). We'd want the machine to make beginner-like mistakes. The first step in "dumbing-down" the AI would be to figure out what kind of mistakes beginners make, and under what conditions. We'd be doing pretty well if we programmed the machine to threaten checkmate when it can be beaten by mate in three. We'd be doing even better if we figured out which kinds of mistakes are more obvious for players (and avoid those). So, the machine wouldn't fall for a sufficiently "obvious" mate in three.
In other words, we'd want a function which evaluates the "cleverness" of a move; the machine, imitating a beginner, is less likely to notice clever moves. Then, when we're setting up a scenario for the player to exploit, we have a believable reason for making the "error."
The AI is not playing the game to win.
The AI is a granny playing with a spoilt brat who throws a tantrum when it loses.
Personally I prefer AI with exploitable weak points, but most gamers are of the spoilt brat variety.
UT used to give the bots fewer shards in the Flak Cannon at lower AI levels, and make the rockets slower... pretty crazy.
We talked about this on the games-and-industry podcast Visiting The Village this week (www.visitingthevillage.com).
For example, take the game Monopoly. there would be specific strategies or "Tendencies" that were predetermined by the programmer to have top priority over other decisions regardless of how useful it was.
For instance, Maud will always buy level 4 properties (The green and blue properties) if able, and place houses on them as soon as possible. This would include a bias of being willing to offer VERY large sums of money in order to acquire there properties or giving up any lesser (according to the bias) property. while at the same time its shunning Level one properties entirely; this would include certain reckless actions, such as trading properties that were not level four even if it owned two out of three of a lesser property of the same color.
This sort of AI gave the illusion that this AI personality had a strategy. you would notice how "badly" he wanted boardwalk and you would hold out for a better offer. You got the feeling you were outsmarting him. and if you picked up on the tendencies, you felt much smarter, and rewarded for victory over them.
I think it would be a nice little tweak to shooter games if personalities were used. Players would know that there were very tactful enemies, as well as overyly aggressive (Reckless) ones. In this instance players can have waves of stupid enemies, mixed with a few, "Hard to kill" Bruce willis types. I think a dynamic AI such as this will enrich most shooters.
Schaeffer recommends that you tune your player so that it is risk-seeking. This doesn't necessarily make it inherently weaker, but if it gives its opponent an interesting game they wil be happier even if they lose.