One of the best ways to give the characters in your scene the feeling that they are part of it, and not superimposed upon it, are shadows. They give more depth to the scene and plant your characters firmly upon the ground. When your characters jump, their shadow and the character part, and greater sense of height is felt. In many cases, shadows have become a standard feature of many a 3D engine.
The iPhone, however, and its hardware limitations, present a problem for the typical 3D engine. At a glance, it would seem that the common techniques used to produce shadows are not possible on the iPhone. Limited fragment processing makes shader-based techniques less than ideal, and often requires a costly memory investment. Shadow solutions requiring dynamic textures are also not ideal, as they also require more memory, which is at a premium on the hardware.
Meanwhile, stencil buffer-based shadows are out as well, as the iPhone has no stencil buffer... Aren't they?
Before we delve into how this might be possible on an iPhone, it is important to recall how the Stencil Shadow technique works. At the highest level, the method is straightforward. First, the scene objects are rendered.
Second, the objects to cast shadows have their edges processed, extruded into volumes, and the volumes are rendered into the stencil buffer. Last, a full screen shadow colored quad is rendered over the scene using the stencil buffer as a mask.
There are dozens of approaches to how to extrude the edges of the objects into volumes, and plenty of other sources on how to do just that. For the purposes of this article, let's assume that we know how to generate shadow volumes, and we have extruded the volumes of our geometry so we can get on with the more important parts of the method: rendering the volumes.
While there are concerns about what to do if the camera is inside the shadow volume versus outside the shadow volume, that's deeper issue than needs to be explored right now. The idea is to render the volumes twice. The first time, we render the volumes with back face culling on, z-testing on, and we add one to the stencil. The second time, we render the volumes with front face culling on, z-testing on, and we decrement one from the stencil.
After both passes, what is left in the stencil buffer is zeros where there are no shadows, and greater than zero anywhere else. Basically, anywhere there are back faces and front faces without other geometry in between, there is no shadow. When there is geometry between the back faces and the front faces, the front faces end up getting rendered but not the back faces, thus leaving the stencil buffer with a positive count.
So how is this accomplished on the iPhone which has no stencil buffer? Many years ago, before 3D accelerated hardware was common on desktops, there existed only fixed function pipelines and hardware with limited functionality. Stencil buffers were considered something only needed by high end hardware, however nearly all hardware had a back buffer that had an alpha channel. This alpha channel was used for many things, and in some cases, could even be used as a stencil.
At first blush, the conversion seems simple from the stencil buffer to the alpha buffer. Stencils can add, alpha buffers can add. Stencils can subtract, alpha buffers can subtract. So for a first attempt, all of the stencil operations are swapped out with similar alpha operations.
The first pass of the stencil buildup process. Negates the initial alpha buffer.
On the first pass of rendering the shadow volumes, back face culling is on, z-testing is on, and we use the Add operation with One for a source blend and One as a destination blend.
The second pass of the stencil buildup process. Darkens the shadowed areas and forces the unshadowed areas to 1.
On the second pass, front face culling is on, z-testing is on, and we use the Reverse Subtract operation with One as a source blend and One as a destination blend. Unfortunately, this doesn't work right out of the box.