Gamasutra: The Art & Business of Making Gamesspacer
The Case for Casual Biometrics
View All     RSS
September 16, 2014
arrowPress Releases
September 16, 2014
PR Newswire
View All





If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 
The Case for Casual Biometrics

December 20, 2012 Article Start Previous Page 2 of 4 Next
 

What is Biometry?

I find it useful, when introducing our work to people who are as yet unfamiliar with biometry, to begin explaining the basic functioning of lie detector machines. When utilizing lie detector machines, the body of a test subject is hooked to several sensors capable of recording changes in a range of physiological processes such as her heartbeat, the electrical conductivity of her skin, the frequency of her respiration, and so forth.

Observing variations in such dimensions, the discipline of biometry is capable of approximating an objective account of the test subject's internal state variations. In other words, by measuring how one's body reacts to a certain experience (which could be a set of questions, or the screening of an advertisement, a video game session, etcetera), we can determine one's level of stress, concentration, anxiety, fear, etcetera.

Whereas traditional quality assurance procedures generate highly subjective answers to research questions, biometry offers a method that produces results that are objective and quantifiable.

By monitoring changes in heart rate, skin conductivity, respiration and the contraction of certain key facial muscles, the lie detector-like setup we utilized was capable of providing valuable insights about game design choices as well as materializing the psychophysiological effects the game has on the players.

Employing biometric experiments and methodologies to analyze a video game, you can obtain scientific answers to several questions that are crucial for its development and commercial success. Examples of such questions include:

  • Is the initial speed of our video game too high for our target audience?
  • Did we reach a climax in emotional involvement where and when intended (this is likely to be at the end of our free demo)?
  • Does the tutorial of our video game succeed in keeping our players engaged while empowering them to perform well?
  • How does our target audience respond to the experience of their first Game Over?
  • Is our game too stressful for them? Is it perceived as too punishing?

The employment of biometric measurements is certainly not a new development in the field of video game design, tuning and testing: Triple-A titles such as Valve's Left 4 Dead and EA Sports' NBA Live 2010 have successfully demonstrated the viability and desirability of biometry as an analytical tool and as a factor of change for their products. Our research project and its benchmark video game Gua-Le-Ni pioneered and optimized the application of biometric technologies and methodologies with the objective of making them available for quicker iterations, exploring their viability as development tools for casual and independent video games.

The Initial Biometric Tests

To complement a wider quality assurance campaign based on questionnaires, interviews, blind-testing and hard-core performance tests, the Dutch research team at NHTV Breda University of Applied Sciences ran an initial series of biometric tests on Gua-Le-Ni. The aim of these initial tests was to structure a testing methodology incorporating the added perspective of biometry.

The first biometric analysis we ran on Gua-Le-Ni focused on its accessibility during the first few minutes of gameplay. The task that was assigned to the researchers was to determine biometrically the optimal speed of the game for the target audience indicated by the developers as soon as the player successfully completed the first tutorial. The game design goal in relation to the initial set of tests was that of achieving the feeling that the game was non-threatening and manageable at the most basic level of difficulty, hence likely resulting in an initially pleasant and positive experience for the casual audience we were developing for.

In terms of game logic, the initial speed of the game is determined by the initial walking speed of the beasts. In this sense, the results of the first test in terms of the walking speed of our bizarre creatures became a cornerstone for all the subsequent design decisions concerning the tuning of the speed and the complexity of the game.

The research team ran parallel tests on two slightly different versions of the game: In the first and harder version, the beasts crossed the screen rapidly (in 24 seconds), while in the second, easier version they would walk from one end of the page to the other more slowly (in 30 seconds). When the testers' stress patterns were analyzed and correlated with in-game questionnaires, we found that -- compared to the harder version -- the easier version showed fewer signs of stress in the participants.

Even in the slower version of the game, however, the recorded stress levels were much higher than expected in comparison with other successful casual games we tested biometrically with the same test subjects. The initial walking speed of the beasts was hence deemed still too high our intended players to simply enjoy the game. We could infer this outcome by combining the biometric data with the self-reported ones. As a result of this observation, the initial speed of the beta game was set -- for the tests that followed -- to 34 seconds. This value was further refined after the second set of biometric tests to the initial speed value of 36 seconds, with which the game was released.

A particular consistency in the Game Over pattern also highlighted difficulties for most testers in recognizing specific creature parts. Responding to the difficulties and the criticism of our test group, the graphics for the creature modules in question were redone.

In addition, we picked the lobster (the beast with the highest rate of stress and failures) as the creature to be used as an example in the tutorial, in order for players to familiarize themselves with its quirky appearance as early as possible in the learning curve of the game.


Article Start Previous Page 2 of 4 Next

Related Jobs

Mobilityware
Mobilityware — Irvine, California, United States
[09.16.14]

Senior UI Artist
Machine Zone
Machine Zone — Palo Alto, California, United States
[09.16.14]

Game Designer
BattleCry Studios
BattleCry Studios — Austin, Texas, United States
[09.16.14]

Economist
Raven Software / Activision
Raven Software / Activision — Madison, Wisconsin, United States
[09.16.14]

Lead Engineer - Raven






Comments


Stefano Gualeni
profile image
Dear Dario, thank you for your kind comment and your interest. In case you are eager to know more, you can find an academic account of our process on our benchmark game at http://www.icemer.com (it is one of the two papers referenced at the end of the article).

As far as hardware goes, we started with this research project roughly two years ago with one of the cheapest set of biometric sensors on the market called Procompt Infiniti produced by Thought Technology: http://www.thoughttechnology.com/proinf.htm

We decided to use a very basic setup, clearly, because the original scope of our applied science and our industry partners was that of working towards the possibility of making our framework and methodologies viable for small developers. Right now, the set costs a little less than four thousand dollars, which is not too bad. :)

The biggest problem that our researchers and technicians had to solve, however, was not related to the sensors or to the creation of a neutral and isolated room to test in. The hardest problems they had to tackle consisted in bringing metrics from the game, game play videos and biometric data together in a single timeline, on a single machine where changes in psychophysiology, game performance, muscle contraction and game events could be assessed and compared.

Without a working framework capable of allowing hardware and software to communicate automatically, it is nightmarish to perform biometric analysis on video games. It was the case of Gua-Le-Ni, when our framework was just at the beginning of development. It took the technical part of our research team more than a year to develop a working and reliable version of the framework. I believe it's safe to say that, in our case, expenses and difficulties did not end with purchasing and setting up the biometric sensors.

Hopefully, commercial set of biometric sensor will soon come with a framework which is easy enough to utilize and obtain answers from. I do not know if we will be able to disclose the software we wrote and the hardware solutions we found, but I presume more technical papers will be published by the more technical people in the research team.

Once again, thanks for your interest.

David Serrano
profile image
Stefano, we know playing games will trigger a physiological responses in (most) players. But the physiological response to a game doesn't necessarily correspond to the player's opinion of the game. Or to their gameplay preferences in general, correct? Raising a player's heart rate or muscle tension could mean he or she finds the game exciting, but it also could mean the game is frustrating them or pissing them off. If I was tested while playing a game on the highest difficulty setting, I know my heart rate, blood pressure, breath rate, muscle tension, cortisol levels, etc... would spike sharply. But this wouldn't mean I found the game exciting or that I was enjoying it on any level. Because in reality, I find that playing games on higher difficulty modes sucks every second of fun, pleasure or enjoyment out of the experience. And my positive or negative opinion of games is largely determined by how well I believe the normal mode difficulty curve has been balanced for the average player the audience, and how well the casual curve was balanced for new or low experience players. So when biometric data is collected, is combining it with, or reconciling it against the player's verbal or written feedback part of the process?

Stefano Gualeni
profile image
Dear David, thank you for the very interesting question and for having shed light on the fact that I have - perhaps - not been thorough enough in the article with regard to our (multi-leyered) process. As you might imagine, writing this article for a non-specific public demanded decisions about what to omit or simplify about the ways in which we gathered, collated and interpreted data.

Your answer could be, very synthetically, found in this sentence of page 2: “To complement a wider quality assurance campaign based on questionnaires, interviews, blind-testing and hard-core performance tests, the Dutch research team at NHTV Breda University of Applied Sciences ran an initial series of biometric tests on Gua-Le-Ni. The aim of these initial tests was to structure a testing methodology incorporating the added perspective of biometry.”

I am thankful for your question because it allows me to elaborate a little more on our work.

So, David, the first thing that you need to keep in mind while reading about our tests is that each subject that tested our game was also exposed to other TWO video games in the same sector and genre of Gua-Le-Ni (casual time-base action video games). In that way we could obtain biometric data about our competitors and have a rough base to compare our game against. However, we did not tell the test subjects that one of the games was developed internally.

On top of that, we administered to every participants mini in-game questionnaires to be filled in quickly between games (normally administered upon ‘game overs’ and to be rapidly filled-in). At the end of each game session with one of the three tested games (ours plus the two control ones), a more thorough questionnaire about the general game experience – also known as a GEQ – was filled in by our guinea pigs.

At the end of the process described above, we would informally discuss with each participants the merits of the games, making notes about the difficulties, the feelings, the interfaces and generally anything they wanted to disclose about their experiences. The interviews mostly focused on Gua-Le-Ni, which (depending on the subject) was either the first, the second or the third game of the series of action-based casual games they played.

Interviews were the most useful for me as a designer, but they were also poorly reliable. As it turns out, players tend to have a very selective and distorted set of memories about their game experience. The specific literature informs us that they can remember very well the beginning and the end of the experience, and perhaps register accurately a particular event that happened during gameplay, but the rest of the playing session is usually vague in their cognition and is mostly re-constructed a posteriori. The vagueness and the cognitive blanks could be filled, in our case, with metrics, biometrics and videos. In that way, we can complement their feelings with an objective tracking of the game sessions from both an in-game performance point of view and a bodily one. Besides, interviews and records of the game states are normally crucial in determining how the bodily signals should be interpreted (or at least suggest a way in which they could be read.

The riddle of the smiles during ‘game overs’ that was cited as an example in the article was solved precisely during informal interviews, where the players specified that the end of their games were received positively. During the interviews they specified clearly that they wanted to keep going on with the game and that the gradual disappearance of the beast in play behind the curled page always left them with the feeling of having ‘almost solved it’, hence their smiles.

The positive valence of those stress spikes remained a mystery until we compared our notes about the interviews with the actual stress graphs. Interesting, isn’t it?

David Serrano
profile image
Stefano - Thank you for taking the time to reply. Yes, it's very interesting because during my career in magazine publishing, we went through a similar period where new techniques and systems were implemented with the goal of reducing or eliminating the level of subjectivity and guess work from creative and technical processes. So I see many parallels between the problems you're addressing with biometric data and the problems other industries addressed through a marriage of statistical process control and science. I think the game industry will eventually create similar systems, standards and procedures. But implementing them will be a slow, tedious process with a steep learning curve. But the end results are absolutely worth it.

Susan O'Connor
profile image
Interesting, thanks for this

Stefano Gualeni
profile image
Dear Susan, thank you for having read and commented my article.
In case any of you were interested in knowing more about our ongoing process, our framework for biometric analysis, or simply feel like meeting up, shaking hands and the like, well... You might be interested in knowing that I will be one of the speakers at the upcoming 2013 Games User Research Summit in San Francisco on March the 26th. (http://www.gur2013.org/)

In case you are planning to attend, feel free to contact me. Also, a new game might be on the way...


none
 
Comment: