[The team at AiLive (co-developers of Nintendo's Wii MotionPlus peripheral) discuss the current state of motion control -- offering specific and relevant feedback about how to properly detect motions, as well as preparing you for the Move and Kinect-powered future.]
Back in 2006 when the Nintendo Wii launched, few realized it would kick-start a new era of motion control in games. Fast forward four years and today interest in motion control in games is at an all time high. Motion controls have widened the demographic for games because they are intuitive, accessible, active and -- most important of all -- fun.
Sony and Microsoft have now unveiled their motion control systems to the world. Responding to current popularity and demand, hundreds of developers are actively engaged in developing motion-controlled games. Game designers are exploring a new degree of freedom in their designs, and inventing new kinds of gameplay experiences.
Creating a motion control scheme that goes beyond the generic "whack and tilt" approach and delivers a fun and engaging gameplay experience is a non-trivial problem. A button press remains the same no matter how it is done, but a "swipe" can be performed in a variety of ways. Motion control schemes are therefore trickier to design and develop compared to traditional button-oriented control schemes.
At AiLive, we have been fortunate to be early pioneers of motion control. During our development of the LiveMove product series and the Wii MotionPlus peripheral, we've been privileged to work with platform-holders, publishers, and game developers involved in creating motion controlled games.
In this article we will draw on our experience and discuss some of the challenges faced by developers. Then we will gaze into our crystal ball and look far beyond the motion control of today. We'll try to extrapolate current trends and predict what genre and mind-bending entertainment the future may hold.
We'll start motionless with our feet planted firmly on the ground and give a brief explanation of the properties of the major motion control systems available to developers today.
You know what this is.
The undisputed champion of accessible gaming for all and the main attraction of countless inebriated parties, the Nintendo Wii is an example of an "inertial sensor"-based motion control system. The Wii Remote and Nunchuk contain accelerometers that measure force along three axes (X,Y,Z). Since force is measured relative to the three axes then, except for special cases, we cannot know the direction of the force in an absolute frame of reference.
In consequence, two moves that look visually identical will generate wildly different data if the player changes their grip, and therefore initial orientation, of the Wii Remote. Game developers therefore face the challenge of controlling for initial orientation when interpreting accelerometer data. In addition, the accelerometer doesn't measure rotations around its axes, so any twists and turns of the wrist during a move get indirectly measured as linear accelerations rather than changes of orientation.
To detect rotations directly you need a three-axis gyroscope, a device that measures angular velocity (speed of rotation) around its respective axes. Say hello to the Wii Motion Plus. In combination with the Motion Plus, the Wii Remote offers 6 DOF (degrees of freedom) motion processing. With a gyroscope, players can finely control their movements, whether flicking a delicate putt as Tiger Woods or slashing a back swing as Roger Federer.
In addition to providing more granular control over movement detection, with the aid of libraries like LiveMove 2, developers for the first time can use the Wii MotionPlus to include short-term 1:1 position and orientation tracking in their games.
Sony's PS3 Eye Camera and Move Controller.
Sony's Move system uses the PlayStation 3 and the PlayStation Eye camera to detect the controller's precise movement, angle, and absolute position in 3D space. The Move controller contains a three-axis accelerometer, three-axis gyroscope, and a magnetometer. Additionally, the RGB LED lit sphere perched on its top is tracked by the PlayStation Eye camera. The size of this glowing sphere, as seen from the camera, gives depth information.
To implement motion tracking without a camera, the position of a controller must be inferred via mathematical calculations on inaccurate data. The calculation errors blow-up over time and therefore motion tracking beyond a couple of seconds quickly loses accuracy. In contrast, obtaining 3D positional data from the PlayStation Eye camera is more straightforward. The motion tracking remains accurate and does not degrade over time.
Microsoft's Xbox 360 Kinect system.
Microsoft has entered the arena with a high-end hardware bundle, called Kinect, which integrates a microphone and depth-sensing cameras. While Nintendo and Sony consider the tactile feedback of a hand-held controller to be central to the game play experience, Microsoft has instead focused on a "controller free gaming and entertainment experience" featuring full body tracking, gesture, and voice recognition.
Kinect includes a RGB camera and an infrared time-of-flight depth sensor that together provide input to software that infers the 3D position of multiple points on the front of a player's body. So Kinect can in principle "see" and track the whole human body.
Tracking multiple body points from a sequence of images augmented with depth information is a more complex problem compared to tracking a single glowing controller held in the hand. In other words, Kinect gains more general tracking capabilities at the cost of more expensive inference.
However, both Kinect and PS3 Move are flexible and open-ended systems. Kinect could be used in conjunction with a held controller, and the PS3 Move could be combined with camera-based full body tracking.
Kinect also processes voice commands, which enables users to interact with the Xbox 360 console in a natural manner without the need to touch a game controller or TV remote.
Now that we've briefly described the primary motion-sensing hardware available today, let's move on to some of the challenges we spoke of earlier that designers face when creating motion-controlled games. We start by talking about a topic that is at the heart of motion control: motion recognition and tracking.