Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
October 25, 2014
arrowPress Releases
October 25, 2014
PR Newswire
View All

If you enjoy reading this site, you might also want to check out these UBM Tech sites:

Interview: Float Hybrid Plugs Into The Matrix With Kinect
Interview: Float Hybrid Plugs Into The Matrix With Kinect Exclusive
September 29, 2011 | By Kyle Orland

Visual effects veteran and innovator John Gaeta says he can't show you everything he and his team at the recently unveiled Float Hybrid is currently working on.

But the stuff he can show you includes some of the most interesting uses of the Kinect's motion-sensing technology yet, and represents the tip of what Gaeta says will be possible with the advanced motion-sensing technology of the future.

Gaeta, who is best known for his graphical effects work on movies like The Matrix trilogy, says he's been interested in branching out into interactive media for quite some time.

Float Hybrid was not established as a game developer -- Gaeta says the company "did not begin with a mandate to do anything in particular," but was created as a place to explore how to "create more depth to an interactive experience overall."

One of Float Hybrid's main focuses, as shown in multiple demos on its YouTube channel, is letting players easily navigate and interact freely in 3D worlds using the Kinect.

For example, the Float Hybrid team has developed a system that allows players to guide an avatar freely using just the upper body -- lean forward to move forward, back to move backward, or twist at the shoulders to turn. Float's demos also show players using outstretched arms to aim and fire projectiles, both from first- and third-person perspectives, and even dueling over a network.

"Everything that you're looking at can be done seated," Gaeta says of the team's control scheme. "One of the abilities of our company is we can scale and tune to a very refined amount of space and movement and still have fluid interaction. We can do everything sitting with minimal motion."

Gaeta says such an understated, direct control scheme is important to attract adult players who aren't going to want to stand up and jump around for long periods after a day at work. Being able to sit also allows for longer Kinect games that can be more understated, at points.

"In a film, you want to develop energy rhythm, you want to create a very careful map of energy flow," he says. "You don't want to be at 11 all the time, you need to create rhythm and cycles. ... The films that are the ones that tend to exhaust you and overwhelm you -- and I've worked on some -- are the ones that are at 11 all the time or 1 for too long. The same goes for when you design a good game, you need to map flow carefully."

Outside of new control schemes, Gaeta's most ambitious ideas have to do with the melding of games and cinema. This doesn't mean infinitely branching, interactive storylines, Gaeta says, but rather the ability to let players inhabit the world of a movie and interact with it.

He recalls spending long days using wide area laser radar to digitize whole mountainsides for scenes in What Dreams May Come. Back then, he "never thought people were going to be given real-time laser radar. Measurement can be the glue of all things. Measurement is the way you couple reality to virtuality."

Gaeta's vision of the future includes films that are captured completely volumetrically, so viewers could watch from the viewpoint of the filmmaker or jump in and explore the movie world from any angle as the universe continues to evolve around them. "If we acquire reality in different ways, one could basically have interactive experiences, not just sculpted performances that are frozen in time, and you can explore the world, and you can play in the exposition of the film."

Gaeta sees the viewers' role in this new kind of interactive movie as "carving one thread" in a living, dynamic universe -- in essence "incredibly sophisticated machinima" where your captured performance is inserted automatically into the recorded performances of the filmmakers. The idea has been imprinted in his head ever since The Matrix movies, and he thinks we're less than a decade from being able to realize it.

Float Hybrid's introductory video, which uses two Kinects to insert a Gaeta avatar into a 3D scene, represents what he calls a "little crude" way of demonstrating this vision. But really realizing the dream is going to require some better technology. Gaeta considers the Kinect the rudimentary "version one" of a motion capture technology revolution that, much like Edison's early moving picture experiments, will eventually cause a sea change in entertainment. "Version 2 is certainly going to be a game changer once the cameras become HD and the depth sensor becomes longer range and high fidelity," he said.

That additional data detail will allow developers to use features like simple face recognition, and even "drivers and filters based on behavior and on identity. There's a lot that can happen and will happen, that's why it's interesting -- from a social point of view, but also from the point-of-view of how you can actually create a much more immersive relationship with what's on screen."

Gaeta also sees miniaturization and decreasing costs leading to a world where more sophisticated Kinect-like technology is integrated into practically everything. "It's going to be merged into TV sets, laptops, cell phones," he said. "It's going to be made modular and smaller. Maybe it'll be wireless, you can glue a bunch around your room, and you have full 360 [degree] sensing. You just have to imagine what the potential is of having full surround awareness of all motion and shapes in a space. It's the building blocks of very serious stuff."

For motion control to really achieve its full potential, though, Gaeta warns that Microsoft needs to open up the technology to all sorts of developers, so experiments with motion sensing can flourish like touch-screen and tilt-sensing apps have on the iPhone.

"My main concern, opening up the marketplace for people to join... making it very simple and very easy for third parties to provide countless experiences that wouldn't be able to make it through a large process that is required to make AAA games or large games," he said. "It'll be as wide as people's imaginations will carry it, but before people will be able to do that, it requires an environment that allows people to come in. It can't be hacker ghettos for that market."

Related Jobs

Red 5 Studios
Red 5 Studios — Orange County, California, United States

Graphics Programmer
Red 5 Studios
Red 5 Studios — Orange County, California, United States

Gameplay Programmer
Gearbox Software
Gearbox Software — Plano, Texas, United States

Server Programmer
Forio — San Francisco, California, United States

Web Application Developer Team Lead


Cheng Ling
profile image
With all relevant respect, I haven't seen anything from Float that I haven't seen in many other places. 3D scanning, using multiple cameras, superimposing the face on a model ... a lot of that stuff was in the Fun Labs apps. Am I missing something?

Biff Zonky
profile image
I think you are missing a lot. I spent a bunch of money on the Kinect and I don't use it that much. The Fun Labs gadgets were little free toys that were difficult to use and get to work. Our family played with them maybe once and they all had a tough time using them. They all looked kind weak and were not much fun. Scan yourself was kinda cool cos it was quick but even that was so-so. This guy's company looks like it's doing some things that look a lot more interesting than scanning in your pillow (badly) and putting googly eyes on it after nine tries. I have not seen any other kinect games that feature you controlling a full character and moving around the landscape or combining mobile with kinect. That stuff looks awesome. I am excited to see what they come out with.

ryder kane
profile image
What I see in some of these you tube videos is looking pretty ground breaking. Not only is there a wide range of emergent types of experiences, there is also better character navigation controls then I've seen anywhere, networked multi player kinects? Mindcraft like instantly build able worlds you can run thru and play in? Haven't see that yet. And the idea of navigating inside of a scene which has the kind of creative insight of folks like gaeta, seems like potential for something big. I want to do all of that shit. my 2cents.

Alex White
profile image
Itís good to see people starting to think of how this kind of technology can be extended into other types of experiences not to mention away from your console. It would be a shame if the luke warm initial showing of xbox Kinect titles deterred people from developing other uses for depth cameras.

There are a bunch of good points here as well, clearly Kinect-like development requires developers to consider things like energy level, consider ways to use gesture technology from a sitting position and explore ideas involving other types of media. Iím shocked there hasnít been more attention paid to integrating Kinect with film and television, gesture technology seems like the perfect fit for more passive media simply because in that format it is less strenuous. Mobile integration is something even more fascinating, Tricorders here we come!

Gesture technology, depth sensing with the ability to generate geometry and superimpose images, this stuff is the future. Whether that future comes in the next couple years or decades down the line seems to me to depend on people finding new ways to use and integrate those technologies. Otherwise these amazing new tools will be dismissed as gimmicks because of unimaginative implementation. If I see another soft on rails kinect action game again it will be too soon.

Vince King
profile image
I'm actually excited to see studios forming up that are willing to take a risk with new technologies and ideas and try to bring them to market. It is extremely difficult in todays climate to find financial backers for anything new and risky. So, I for one wish the guys at Float (and any other studios in this space) the best and can't wait to see what they all bring to market.