As a result of this limit of game AI, I automatically assume it won't be convincing and forgive any errors it makes, such as running into things, repeating itself, taking unnaturally long pauses during conversation, and staring at me. Fragmented AI regularly communicates its inhumanity and punctures immersion.
However, it is becoming increasingly sophisticated, and that means that as it engages more of the parts of our brain used in socialization, it will pass a point where it will stop looking like good AI and start looking like bad acting or dysfunctional behavior. When interactive entertainment hits that point, it won't just be something we can laugh at like a B-movie, because it won't be a passive experience. It's going to be reaching out to us and pushing all the wrong buttons.
There are limited examples of it happening already. In Half Life 2, Alyx being programmed to look at the player while talking to them to create a sense of eye contact was a step above the previous generation of art and AI, but the illusion snapped when she was talking to me on a descending lift: Her eyes kept slowly rolling upward then flicking back down to me, because the point she was scripted to look at wasn't updating as fast as my location. If a real human did that near me, I'd be concerned for their well-being.
In that game, despite every emotionally convincing moment delivered by the combination of story telling, AI and art assets, it only took that one error to unhook a great big wedge of my empathy and make me laugh.
The closer a representation of a human is to reality, the slighter the flaws that can suddenly de-animate it. AI systems are rather fragile right now, whereas organic intelligence is decidedly robust, being able to operate in and adapt to a multitude of contexts.
We tend to take our own adaptivity for granted because it's such an everyday thing, and it's often the oddities of humans that make them more interesting and charming. Only certain subsets of characteristics make socialization more challenging, and even then it can be offensive to define them as flaws.
Sometimes it's just that people are a little de-socialized, but even so I think an important and much more formal connection between people and present level game AI can be found in psychiatry: the autism spectrum.
This spectrum is a psychiatric construct that defines various behavioral symptoms as disorders, varying in severity. Stated very simplistically, some positions on the spectrum involve enhanced specialization and lack of social ability, but it should be stressed that this is not a trade off.
The extraordinarily talented autistic savants sometimes paraded on TV and brought especially into public consciousness by the film Rain Man only comprise a fraction of autistic people. Also, while it is well known and obvious that they have limited social ability, an incredibly important component of autism is rarely discussed in popular culture: The ability of autistic people to understand the subjective viewpoints of others is drastically impaired.
To illustrate, an autistic child is shown a model in which person A puts an object away in front of person B, then leaves the room. Person B then takes the object and conceals it in a different location, then person A re-enters. If told that person A wants the object back and asked to show where he will go first to get it, an autistic child will likely point straight to the hiding place used by person B.