Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
May 12, 2021
arrowPress Releases

If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Can The Xbox One's Kinect Read Your Mind?

by Jamie Madigan on 07/07/13 07:30:00 pm   Expert Blogs

6 comments Share on Twitter    RSS

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.


Well, no. Of course not. That's a silly question. Why would you even ask it? 

The updated supercamera on the Kinect 2.0 has been shown to be capable of some pretty amazing things, though. Microsoft demonstrated how it can tell where you're looking, estimate your heart rate from the color of your skin, and even infer your mood from your facial expressions. Finally, it has a sophisticated voice recognition system and the ability to see in the dark, which will come in handy when it wants to sneak into your bedroom at night and listen to your breathe for hours on end.

And though it hasn't been discussed, I wonder if the Kinect 's high definition camera could be programmed to measure one other important biometric: pupil dilation. This would be both awesome and worrisome, because while not exactly a mirror into our souls, the eyes can reveal a lot about what goes on in our minds. 

How Kinect sees you: a pulsing sack of meat and emotions.
 (Image from Wired's Kinect demonstration.)

Daniel Kahneman wrote  in his book, Thinking Fast and Slow, about how pupil dilation is a darn good proxy for mental effort. In a series of experiments he asked people to take a multidigit numer, then increment each digit in the number by one to form a new number. So 348 would become 459. The would then do the same to a new number, using a metronome to do one new number/sum every two seconds.

Try it yourself and you'll see that the task gets very difficult pretty quickly. If not, try adding 2, 3 or even 4 to each digit instead of 1, BRAINIAC. And if you had someone eyeball your eyeballs they would clearly notice that your pupils would steadily grow larger and larger as the mental machinery behind them started to work harder and harder --right up until the point where you gave up, when they would snap back to normal size.

In what sounds less like science and more like an exhibit you'd make a face at while visiting the museum of contemporary art, Kahneman and his colleagues would train a camera on subjects during these experiments and broadcast an enormous image of their eye onto a television in the hallway so that the pupils were about a foot wide and dilation was easy to measure. The results were pretty consistent: the more mentally taxed we are, the bigger our pupils get.

If the Kinect (or any camera) could detect pupil size, it would open up a whole new level of scaling game difficulty. A puzzle game could be made more and more difficult until you're taxed just the right amount to get you in the zone --something psychologists call "psychological flow." Imagine playing a rhythm game like Guitar Hero and having the game adjust the speed of the note highway until you're pushed just to the brink of your abilities based on how hard you're concentrating on the task. 


Or what about knowing when to offer you a helping hand? If the Kinect can tell the point at which you've given up on a puzzle or sequence because your pupils shrink back to normal, it might offer you a hint. Possibly in a condescending tone.

Another, more unsettling implication would be that if the Kinect could tell when you are stressed and mentally taxed, it could use that opportunity to sell you stuff. Willpower is like a muscle that can be exhausted by any mental activity, and when it's depleted we're more likely to do dumb stuff like make impulse purchases or, one might imagine, place an impromptu Skype call to an ex-boyfrined when we really should know better.

Jamie Madigan writes about the psychology of video games. Read more at and follow him on Twitter or follow him on Facebook.

Related Jobs

Visual Concepts
Visual Concepts — Austin, Texas, United States

Sr Designer
Insomniac Games
Insomniac Games — Burbank, California, United States

Design Manager
Visual Concepts
Visual Concepts — Austin, Texas, United States

Visual Concepts
Visual Concepts — Austin, Texas, United States

Sr Artist, UI

Loading Comments

loader image