Do you see much in the way of middleware that actually helps what you do on the graphics side?
HC: Absolutely. We see more and more roles played by middleware these days. We ended up using a few middleware packages ourselves, even though we wrote the rest of the engine. I think good use of middleware allows us to get a feature earlier than we could if we were to engineer it ourselves, and that means the content people will get mature technology and more time to produce content. So that longer iteration time will translate to higher quality that will offset whatever engineering advantage you would have by writing your own. So in a lot of these places where it makes sense, we use middleware.
My graphics programmer friends are curious to compare how you spend your day-to-day. Are you mostly writing shaders, or doing engine development, or do you develop tools for the artists?
HC: For a typical graphics engineer [at Bungie], we have about 65 or 70 percent of the time devoted to coding. The rest of the 30 to 35 percent is spent talking to the artist, educating the content people how to properly use the tools, and gathering feedback into what needs to be changed.
So we spend about 30 percent of that time doing these tasks. The rest depends on the phase of the project we're in. In the beginning it's very much just core development of the technology itself, but if the feature comes to maturity, part of that coding time is devoted to bug-fixing, optimization, and also tools development.
Do you work on more tools development on the front end to get people implementing stuff faster?
HC: Well, we'd like to. The ideal is before you develop a feature, you figure out exactly how the content people will use it. That's the most ideal case, because then the content people have the maximum amount of time they can spend to get familiar with the tools and give feedback.
But it's quite often not that ideal, because we don't know what kind of feature we need to expose until we have enough of the technology to give them a taste of what this feature is about. Then they will say, "Hey, I really want this and that," and that will factor into the tools.
So in the beginning we typically come up with very straightforward, programmer-ish tools, and we just give them to one or two technical art leads to try it out. But before that feature goes to primetime, that's where we spend a lot of time making a very professional tools interface, so the bulk of the artists will then be able to use them.
This is a broad question, but what do you think are the big problems to solve in the coming years from your perspective, graphics-wise?
HC: That definitely is a big question! I will tell you what our emphasis is for what we think is important for our graphics and our engine. Number one is removing digital artifacts. You mentioned this already... removing all the jaggies, having very clean foliage edges, and awesome looking hair with no artifacts. Removing these digital artifacts that remind people you are staring at a computer screen is one of our top priorities.
The other challenge is selling a dynamic world. In terms of what we think is important, we will even lower some of the quality in order for us to have a more dynamic world. This means dynamic time of day, lots of things that move in the wind, lots of things reacting to players moving through them, and when you walk on soft surfaces like sand and mud, you leave footprints. So basically everything we do to sell that this world is moving and dynamic is important to us.
And then perhaps the last one is believable characters. That's still one of the areas where we still spend a huge amount of emphasis in animation, in the rendering of character faces and facial animation, and just characters in general, that's still one of our high emphases. In the end what we're trying to do is deliver the fidelity where it really matters to the end user.
All of the stuff that I just mentioned to you is because that's the stuff that makes the player not believe they're in this game. If you see jaggies, if you've seen things that should be moving but are static, if they see a character's face and they don't believe it's a real face, then their illusion is broken. So we want to spend all of this energy to try to remove these things and deliver the fidelity where it really matters.
How far away are we from having realistic hair? I know that's one of the big reasons that we have so many bald space marines in games right now.
HC: Hair is interesting, because it's actually kind of easy to do very long, natural, flowy hair, because of tessellation. We can do lots and lots of strands of straight hair. With a good material map to it, and a good lighting map to it, it looks very real. And it moves very real, because we have plenty of computing power to do the simulation part. But the problem is stylized hair. So we're talking about making many different styles, from a buzz cut to choppy hair to curly hair. So those are still very difficult to do and to make look real, so it will continue to be a difficult problem I think.