Hardware Accelerating Art Production

The use of shaders is not only impacting real-time rendering in a fundamental way but also game production in terms of workflow and art tools as we seek to maximize their visual potential. With more parameters and surface maps affecting the final on-screen look of game characters and scenes, it's more important than ever that artists have tools which enable them to fine-tune their work quickly and easily.

Thankfully, the major 3D graphics packages are adding native support for shaders. But in-house art tools (such as custom plug-ins, editors and build processes) should strive for the same level of usability.

This feature promotes the use of programmable graphics hardware outside of real-time rendering, in the processing of artistic effects in game development. To demonstrate the power of this technique, a large chunk of this article concerns mapping an intensive precomputed shadowing technique for meshes (ambient occlusion) onto the GPU. The method can be 15 times faster than a solution that takes advantage of only the host CPU.

This technique is a boon for art builds, but it also opens up the possibility of wrapping things up in a plug-in that would let artists effortlessly generate, visualize and tweak vertex or texture data within their 3D art package. In addition, the implementations presented here use version 1.1 Direct3D shaders for processing, so they can be widely deployed for use with today's common graphics cards.

Feature Overview

The first section looks at how technological advances affect art production and what features tools need to provide. An overview of ambient occlusion follows, leading to a three-step workout, turning an initially sluggish hardware approach into a lean, streamlined solution.

In fact, several variations on the technique are covered, illustrating a range of production trade-offs. I'll also show how these can be extended to handle higher-order occlusion (generalized precomputed radiance transfer) before closing with a round up of other accelerated preprocesses that you might wish to consider.

Art Production

To begin, let's consider the broad picture of art production and where ambient occlusion (AO), as a preprocess, fits in. The technique for generating the AO shadows can be integrated into the creation end of the production process--3D art packages such as 3ds max and Maya that are already using real-time shaders, giving the 3D artist an instant preview of what the effect will look like. Of course, it could be integrated with any commercial or custom tool that has access to 3D mesh data.

The Case Study

The AO process (for game purposes, at least) takes in one or more meshes and spits out vertex or texture data. There are a number of variations described later on, but with the basic technique this data consists of a single shadow term for every surface point--stored as an additional vertex attribute or packed into an occlusion map--which is used in-game in addition to ambient (or diffuse environmental) lighting.

In contrast to its low run-time cost, the processing is pretty intense, since lots of visibility samples need to be taken at each point. With a typical CPU-based approach, AO forms part of a lengthy build process that is far from ideal for editing and previewing purposes. Speeding up the calculation will not only yield faster builds but also afford us the opportunity to expose AO as a modeling package plug-in, so that artists can iterate more readily on content.

I will provide an overview of AO, but the subject is covered in greater detail by a chapter in the forthcoming GPU Gems book (see "Additional Reading"). Whether you're already familiar with the technique or not, this primer sets the scene so to speak for mapping AO onto graphics hardware.

Ambient Occlusion: A Primer

AO measures the amount that a point on a surface is obscured from light that might otherwise arrive from the outside. This average occlusion factor is recorded at every surface element, vertex, or texel, and used to simulate self-shadowing (see Figure 1).

	Figure 1: Breakdown of Ambient Occlusion in conjunction with environmental lighting: (left) Mesh wrapped with an occlusion map, (middle) Lit via a diffuse environment map (in this case from a photographic source), (right) Lit through a combination of the environment and occlusion maps.

The extra soft self-shadowing adds a great deal of believability to the lighting.

Note: This example setup is available as an effect which comes bundled with the RenderMonkey shader development suite.

The technique has come to prominence through its use in the film industry, with ILM first employing it for Dinosaur [Landis02]. Its niche is attenuating soft environmental lighting, particularly from indirect sources such as walls, sky, or ground, achieving the look of more complex setups--usually the manual placement of extra bounce lights or full-blown global illumination--at a lower cost.

One point that should be stressed is that fine-tuning lighting is a frequent task in post-production, and by decoupling the shadowing and lighting, it is possible to tweak the lighting without re-rendering the shadows.

Assumptions

Several assumptions are required for the re-use of visibility information to work:

• Diffuse surfaces
• Rigid geometry
• Constant lighting

The first two constraints are also required by traditional precalculated radiosity. The diffuse surface limitation means that the stored visibility is view-independent, and in conjunction with a lack of deformation, ensures that the data can be re-used. Because of the separation of lighting and visibility, AO is less restrictive than static lighting since single meshes with independent occlusion can be merrily translated, rotated and uniformly scaled. A group must be transformed as one however, in order to preserve inter-object shadows.

	Figure 2: The two basic forms of AO: (left) Vertex data - note the linear interpolation artifacts behind the ear due to the low tessellation, unable to accurately capture shadow changes in this region. (right) Texture data - note the smoother shadowing.

The Process

Equation 1, expressing occlusion, O_p, in integral form, is typically solved via Monte Carlo integration. For detailed coverage of the methodology consult the "Additional Reading" section.

	Equation 1: Hemispherical integration of visibility

Rays are traced outward from a given surface point p over the hemisphere around the normal N. A binary visibility function V is evaluated for each of these rays, which returns 0 if the ray intersects any geometry before reaching the extent of the scene, and returns 1 otherwise. Figure 3 shows a simple 2D depiction of the process for a single semi-occluded point. In reality, creating smooth gradations in the shadows requires hundreds or even thousands of rays.

	Figure 3: A cross-section showing visibility sampling over the hemisphere for a point p. Many rays are fired off and tested for intersection with blocking geometry, with the results averaged together.

In the case of a uniform distribution of ray directions, each visibility sample is weighted by the cosine between the normal and sample direction (again, consult Additional Reading for an explanation) and averaged together, resulting in our scalar occlusion term. For efficiency, ILM uses a cosine distribution of rays, which removes the need for the cosine factor (as shown in Equation 2) and concentrates samples in statistically important directions.

	Equation 2: Monte Carlo integration using a cosine ray distribution

Game Use

For games, AO immediately enhances constant ambient lighting, adding definition to otherwise flat, shadowed areas. NVIDIA's Ogre demo (see Figure 4) uses AO in this way, in addition to a key directional light plus shadow map.

	Figure 4: Break-down of lighting in NVIDIA's Ogre demo: (above) AO (below) AO multiplied by constant ambient lighting, on top of key lighting (directional light plus shadow map). Images courtesy of Spellcraft Studio GmbH and NVIDIA Corporation.

This is the only correct use for AO, where lighting is assumed to be constant and thus it can be factored out. In practice, however, we can stretch things slightly as the overview suggested. ILM, for instance, uses a pre-filtered environment map for secondary lighting of diffuse surfaces, instead of a uniform ambient term. Here, believable results are achieved from shadowing with AO because the illumination varies slowly. They also improve on the approximation by storing the average un-occluded direction--a so called "bent normal"--which replaces the surface normal when indexing the map.

Spherical harmonic (SH) lighting [Forsyth03] is often a more attractive option than using a pre-filtered map. Again, AO can be used for attenuation but it can also be extended to precomputed radiance transfer (using SH), as is shown later.

In summary, AO is a cheap but effective shadowing technique for diffuse environmental illumination, and the combination of the two can add a plausible global effect on top of traditional game lighting. Using the power of today's programmable graphics cards, we can accelerate the technique enough to make it usable during the creation of 3D models and art, as described next.

Related Jobs

Square Enix Co., Ltd. — Tokyo, Japan
[09.23.21]
Experienced Game Developer

Insomniac Games — Burbank, California, United States
[09.23.21]
Lead Gameplay Programmer

Playco — USA, Remote, Remote
[09.23.21]
Senior Game Engineer

Insomniac Games — Burbank, California, United States
[09.23.21]
Technical Artist - Pipeline

130455 item_feature /view/feature/130455/hardware_accelerating_art_.php Loading Comments

---