[Editor's
note: this article on multiple texture compression techniques, written
from a programming perspective, should ideally be read in conjunction
with the recent article by Riccard Linde that discusses the issues of DXT compression from an artist viewpoint.]
As
games become more and more visually complex, so does the artwork that
is needed to create such detailed environments and objects. The more
detail that goes into a piece of art, such as a texture image, then the
more space that is needed for us to store such a resource in memory.
With our resources becoming larger and more space-consuming we are
required to have more bandwidth with our hardware in order to push our
resources down the rendering pipeline. What we need is a way to take
our high resolution artwork and reduce the memory requirements needed
to store and work with the information.
In
this article, we will look at a few different texture compression
techniques that can be used to do just that. These techniques each have
their own strengths and weaknesses, and it is important that we have a
clear understanding of these before choosing a texture compression
format that will be used for a set of texture images. Some formats only
work with normal maps, while some work for both decals and normal maps.
To further complicate matters, some formats can have an impact on the
visual quality of an image resource, which (as we will see later on in
this article) is not a good thing when dealing with normal maps.
Introduction to Texture Compression
The
amount of artwork in a modern video game is tremendous when compared to
games of the past. The texture images that are being used are higher in
resolution, numerous in numbers and heavy with detail. Although
graphics hardware has come a long way and offers us a lot of memory, we
still don't have enough space to the point where we can load images
without the concern and issues that arise with running out of memory.
Techniques
like bump mapping, normal mapping, etc further complicate things
because 1) the resolution of these images are often pretty large in
size and 2) normal maps created from geometry data have to be huge in
order to fully cover a model object with a decent level of quality. In
order to make the most out of the memory we have available to use, we
must use image compression in our video game applications. By
compressing texture images we can reduce the amount of memory that each
image requires on our hardware and, in some cases, we can use the
compression techniques to increase the visual quality of normal map
images. Using compressed textures can also boost an application's
performance, since we are not required to send as much data down the
pipeline.
The
purpose of this article is to take a look at a few different texture
compression techniques that can be used to reduce the size of the
images that are used in a video game application. We will be
particularly looking at the A8L8, DXT and 3Dc compression formats, as
well as how each format stacks up against the others when dealing with
decal textures and normal map images.
Each
texture compression format that will be discussed in this article can
be applied to both Direct3D (the API in the DirectX core that is
responsible for rendering 3D graphics on the Windows OS) and OpenGL (a
platform independent graphics library) with both graphics APIs having
the ability to store texture images in these formats directly. In
addition to that, we will also be looking at how we can use OpenGL to
take uncompressed texture images loaded from a file and allow the API
to perform the compression internally. We can then take that compressed
data and save it out in a file so that it can be used later on (i.e. we
can save the compressed image out so that we don't have to waste
processing time compressing it again every time the application
starts).
Texture Compression Algorithms
There
are many texture compression algorithms that we can use in our games.
As previously mentioned, this article will look at the DXT (S3TC) and
3Dc texture compression formats as well as the A8L8 format. Each format
has its own strengths and weaknesses.
For
example, the 3Dc compression format is great to use for normal maps,
but the DXT compression formats are not so great when dealing with
non-decal textures. The DXT1 compression format allows for a 6:1
compression ratio while the DXT2 / 3 / 4 / 5 and 3Dc formats allow for
a 4:1 compression ratio. A8L8 is not technically compression but it can
allow us to represent data using two components instead of three. For
normal maps this means we can simply calculate the Z axis in a fragment
program and save quite a bit of space when storing our image data.
Below we will take a closer look at each format one by one starting
with the DXT (S3TC) compression formats.
After
we have a look at each of the compression formats described in this
article, we will look at how we can generate the Z axis of a normal map
in code. Not only does this allow us to only need the X and Y (red and
green) axes values of the normal texel but it also means that we can be
sure that all of our normals are unit-length. Since a unit-length
normal has a length that is equal to 1, this is easy to accomplish.
DXT Compression Format
The
DXT compression formats use a lossy compression that can reduce an
image's size by a ratio of 4:1 or 6:1. The DXT formats are a standard
part of the Direct3D API and are available for the OpenGL API through
the ARB_texture_compression and GL_EXT_texture_compression_s3tc
extensions.
The
DXT compression formats are made up of DXT1, DXT2, DXT3, DXT4 and DXT5.
DXT1 gives us the most compression by using 4-bits for each pixel but
does not require an alpha channel (if it has one then it is 1-bit).
DXT2/3 is the same as DXT1 but it uses an additional 4-bits for the
alpha channel, thus doubling the size of the image. In the DXT2 format
the data is pre-multiplied by the alpha channel while in the DXT3 it is
not. The DXT4/5 formats are similar to the DXT2/3 formats with the
exception of interpolating the alpha data when compressing the images.
DXT2 / 3 / 4 / 5 can give us a compression ratio of 4:1 while DXT1 can
give us a ratio of 8:1 / 6:1 (8:1 if we are not using the alpha).
The
DXT compressions are good to use on decal texture images, especially
those images that are high in resolution. The problem with the DXT
compressions is that they were not designed with normal maps in mind,
and can have horrible results when working with those types of images.
Take a look at Figure 1 for a comparison of an uncompressed decal
texture with a compressed decal texture. After looking at Figure 1,
take a look at Figure 2 for a comparison of an uncompressed normal map
and a compressed normal map.
|
|
 |
 |
 |
Figure 1 (Left)Uncompressed image, (Right) compressed image.
|
|
|
 |
 |
 |
Figure 2 (Left)Uncompressed normal map, (Right) compressed normal map.
|
As
we can see from Figure 1 and Figure 2, the DXT compressions can do a
pretty good job compressing decal texture data, but does not quite get
the job done when trying to compress normal map data. This is due to
the fact that the DXT compressions assume that the images that are
being compressed have smooth changes (as we can see in the normal map,
this is not always the case) and it using 16-bit reference color values
which are not enough to represent all of the possible normal
directions. Using specular lighting with normal maps compressed using a
DXT format can increase the artifacts thanks to the power function used
in the calculations (the power function amplifies the artifacts).
The
DXT formats are good to use for non-normal map data but, with a few
tricks, we can use DXT compression with them if we really needed to.
These tricks include re-normalizing the normals for each texture fetch,
placing the X axis of the normals in the alpha channel when using DXT5
compression (since the alpha channel is compressed separately, we can
get better results than with the DXT1 compression), and generating the
Z component and placing the X axis in the alpha channel when using DXT5
compression.
For
the last trick, since the alpha channel is compressed separately and
has 4-bits all to itself, we can store a higher range of values inside
of it. By generating the Z axis we can ensure that we are working with
unit-length normals and we don't fully need to limit ourselves to what
can be stored in the compressed image data. This allows us to use the
DXT5 compression format with normal maps at a higher quality than
before, but it still has problems and is not perfect.
3Dc Compression Format
The
3Dc compression format is a 2 component format that was created by ATI
as an effective means to compress texture data. The 3Dc can be used
effectively to compress normal map data or to compress multiple pieces
of data into a single texture. The problem with the DXT compression
format is that DXT will cause heavy artifacts in the normal map image
because it can not capture the small, fine detail (such as the
curvature or small edges of the normal map) effectively. Since the
purpose of normal maps is to capture that detail in an image, this can
cause some huge artifacts in our rendered scene. These artifacts can be
so bad that it can have a huge negative impact on the overall quality
of the scenes that are being rendered.
The
purpose of the 3Dc format is to compress our images while retaining the
highest level of detail as we can. The 3Dc compression can allow us to
compress images at a 4:1 ratio. That is like getting four images for
the price of one! The compression is also hardware accelerated so there
is very little performance impact on the applications using this
technique when it comes time to uncompress the data. Take a look at
Figure 3 for a comparison.
|
|
 |
 |
 |
Figure 3 (Left) Uncompressed, (Middle) DXT5 compressed, (Right) 3Dc compressed.
|
A8L8 and V8U8 Formats
The
A8L8 format is a two component 8-bit unsigned texture format (the
signed version of the A8L8 is the V8U8) used in Direct3D. The OpenGL
version of the A8L8 is exposed through the GL_HILO8_NV that is part of
the NV_texture_shader3 extension. By only using two components we can
not use this format for decal textures because there is no way for us
to know what the third component (blue component) is supposed to be.
Since
this format is not used for decal images, it is used to store luminance
images with an alpha component (the luminance value is stored in the
first component while the alpha is stored in the second) and it is also
used to store normal map data. It can store a normal map by using the
two components to store the X and Y axis of the normal for each texel.
Since normals are unit-length (i.e. all components squared and added
together equal 1) we can just use simple mathematics to figure out what
the Z axis of a normal should be since we know what the X and Y axes
are.
Although
this format does not technically use compression it does allow us to
take an uncompressed normal map and store it using much less space by
dropping a component entirely. Since we are dropping a component from
each texel we are going from 3 bytes per-texel to just 2 (i.e. 1/3 less
space). In a fragment shader, we can just calculate the Z axis of a
normal quickly and easily.
Generating the Z axis for a Normal Map
Since
unit-length normal components all add up to equal 1 (R * R + G * G + B
* B = 1.0) we can use some high school algebra to create a “solve for
x” type equation as see below:
(A * A) + (B * B) + (X * X) = 1
For
example if we know A squared (red component) is equal to 0.1 and if B
squared (green component) is equal to 0.3 then the equation becomes:
0.1 + 0.3 + X = 1
0.4 + X = 1
X = 1 + -0.4
X = 0.6
In
code (for example a fragment program) we can calculate the Z axis of
any normal by using just the first two axes of the normal. This can be
done like so:
Z * Z = 1 – X * X - Y * Y
Or
Z = sqrt(1 – X * X – Y * Y)
Since
the normal is always unit-length, the results of using our normal map
in this format can give us better results if we choose not to
renormalize normals in the fragment program each time the program is
executed.
Summary
The
added benefits to using texture compression are huge, and it is
something worth looking into when working with video games. When
working with 3Dc compressed textures we can get four times the detail
out of our normal map images without sacrificing performance or
requiring more memory on our hardware. The DXT compression formats are
great to use when dealing with decal texture images, but are not the
ideal choice for working with normal map images. Using DXT compression
on normal maps will alter the normal data that was encoded in the image
enough to the point to where our visual quality will be impacted for
the worst. This is due to the format's lossy compression scheme, which
is not anything we want to apply to normals in an normal maps.
If
speed is an issue, we can also use the A8L8 and V8U8 formats. These
formats require 2 bytes per-texel, which is twice as big as the DXT
formats which require 1 byte for each texel. By not using the DXT
formats, you do not have to worry about the lossy compression altering
the original quality of the image, and the hardware does not have to
spend time uncompressing the data when we try to access it in a
fragment program. If A8L8 is used for normal maps then the Z (or blue)
component of the image must be calculated in a fragment program.
Regardless
of how we look at it, texture compression will probably always be
something that we will have to work with as games become more and more
complex and the artwork for our games increase. Which texture format
that is used will depend on the type of images we are working with
since some formats work much better for some types of images (for
example decals vs. normal maps) than they do for others.
For
more information about the texture compression formats described in
this article, please check out the resources section at the end of this
article. When using one of the formats described in this article keep
in mind the follow:
- When using A8L8 we can't use them for decals because there is no way to guess what the blue component is.
- When
linear filtering normal maps the filter can cause normals to become
less than normalized. If this happens then we will have to renormalize
the normals in a fragment program each time a texel if fetched.
- Artifacts will arise when trying to use normal maps and a DXT compression format, especially with specular highlights.
- Generating the Z component can increase the rendering quality.
- The 3Dc compression format is better for normal maps than the DXT compression formats.
- DXT1 gives us the highest compression ratio.
- There
is a performance gain when working with compressed textures because
there is not as much data that needs to be sent down the pipeline.
- Each format described in this article can work on OpenGL or Direct3D.
Resources
For
more information on texture compression please feel free to take a look
at aNVIDIA technical report titled Bump Map Compression on their
developer website: http://developer.nvidia.com/object/bump_map_compression.html.
Additional information on the 3Dc texture compression algorithm can be found in an ATI white paper titled 3Dc White Paper: http://www.ati.com/products/radeonx800/3DcWhitePaper.pdf
_____________________________________________________