Present & Future

Does it scare you when you see Crysis... or, I don't know. What's the biggest competitor in the game engine market? Is it id's stuff, or is it another company?

TS: Today, it's Crysis. Because Crysis is doing some things on high-end PCs that we're not doing ourselves. That's a tricky case though, because we could do vastly more than we're currently doing if we focused on supporting dual high-end video cards, which have about 10 times the graphics horsepower of a console today.

The thing is, that market is about 2% the size of the overall next-generation game market -- PS3, Xbox 360, and mainstream PC. So there's a real hard business decision: if you go the route that Crytek goes, you can beat us in certain areas in graphics, but you're really sacrificing the larger market.

Because you can't run that engine on the PS3 and that sort of thing.

TS: Yeah. And some of the things they're doing are just conceptually incompatible with that level of performance. Wherever you have an order of magnitude performance difference, you can't really scale.

We can scale down in performance by a factor of three by going to a low resolution, dropping some textures, and things like that. But to scale by a factor of 10 -- you can't design a game with 10 times the detail and then scale it back to something that looks decent on the consoles. You'd end up looking much worse than a console game that was just designed for the console specs. So they have real scalability difficulties there.

On another note, LittleBigPlanet reminds me of ZZT, because you can build your own levels, and people do emergent techniques with a simple set of tools. Have you played LittleBigPlanet? Do you like it?

TS: Yeah, it's brilliant. If I were creating a game with a really small team so that things like Unreal were ruled out, it's exactly the kind of game I'd want to create -- a side-scroller with really cool graphics using 3D and really cool physics -- and design it around the user community building stuff. It's a really brilliant project.

I think it's indicative of the direction in which that kind of game could go -- a sandbox game like The Sims or anything else. You can really go a long way with the simulation without creating complicated tools. It's not like there's some crazy Python scripting language in there; it's all done with really simple graphical stuff. It's a great idea.

Looking ahead, how long do you think it will be before real-time computer graphics are 100% realistic like a movie?

TS: There are two parts to the graphical problem. Number one, there are all those problems that are just a matter of brute force computing power: so completely realistic lighting with real-time radiosity, perfectly anti-aliased graphics, and movie-quality static scenes and motion.

We're only about a factor of a thousand off from achieving all that in real-time without sacrifices. So we'll certainly see that happen in our lifetimes; it's just a result of Moore's Law. Probably 10-15 years for that stuff, which isn't far at all. Which is scary -- we'll be able to saturate our visual systems with realistic graphics at that point.

But there's another problem in graphics that's not as easily solvable. It's anything that requires simulating human intelligence or behavior: animation, character movement, interaction with characters, and conversations with characters. They're really cheesy in games now.

A state-of-the-art game like the latest Half-Life expansion from Valve, Gears of War, or Bungie's stuff is extraordinarily unrealistic compared to a human actor in a human movie, just because of the really fine nuances of human behavior.

We simulate character facial animation using tens of bones and facial controls, but in the body, you have thousands. It turns out we've evolved to recognize those things with extraordinary detail, so we're far short of being able to simulate that.

And unfortunately, all of that's not just a matter of computational power, because if we had infinitely fast computers now, we still wouldn't be able to solve that, because we just don't have the algorithms; we don't know how the brain works or how to simulate it.

So you'd have to create a perfectly realistic virtual human first to have perfectly realistic graphics.

TS: Yeah, you'd have to simulate the brain and nervous system in the computer.

And circulation and everything. But that's probably going to be possible some day, don't you think?

TS: Some day, yeah. But there's no Moore's Law for that stuff, and progress is very non-linear. Somebody must have a clear understanding of how a neuron works now and how it transmits to adjacent neurons, but they have no idea how a billion neurons combine together to create a brain and what parts of our brain are basically hard-coded by evolution, and which parts are based on learning, and so on.

And if you could simulate it all, how could you train it to be realistic like a human? Those problems are probably decades away from being solved. Those are things that may not occur in our lifetimes.

Just like perfect computer speech recognition: if you look at speech recognition, it's only gotten slightly better in the past decade, just by a factor of several hundred increases in computing power. That shows that those problems are not falling to brute force.

They're not following Moore's Law acceleration. Have you ever read any Ray Kurzweil's futurism stuff?

TS: Some of it. Definitely anything like that is long-term future. It's just because we lack the algorithm and knowledge of how to model that, even given unlimited computing power.

Do you think that if the graphics were perfectly realistic that you would create a "final" version of the Unreal Engine? As in, you'd never have to make another one because you got it perfect?

TS: I don't think we'd ever be done. If you look at special effects or anything like that, even if you have perfect rendering, you might have the inner loop of the renderer which processes sub-pixel triangle primitives and generates perfectly anti-aliased graphics -- that might be completely done, but still, the tools and the functionality that lets the artist create the environment is infinitely improvable.

Currently, our approaches are somewhat primitive if you look at it. If you want to build a city for a massively multiplayer game, the artists model a single column or wall, then they copy and paste it throughout the world. You'd run really large-scale procedural tools for defining road systems and defining buildings, and the structure and architecture of buildings.

You could really do a lot more to improve productivity than any current engine or modeling tool is doing. So I see decades and decades of work there, just to make ever-better tools. And special effects as well. Particle systems are only partly limited by computing power.

The other limitation is the tools that the artists have to control what's really going to happen, and to do that productively. If you can make tools more productive so an artist can do more in an hour, then that's an improvement too.