Introduction

Anybody who has tried to display computer graphics on a TV knows that they look bad. Colors wash out, detail is lost, and the whole thing flickers. That's what happens when you're lucky. sometimes the graphics come out really badly.

While console game creators know that TVs do bad things to computer graphics, few of them are sure exactly what is happening. Sometimes "avoid bright red" or "don't use thin horizontal lines" is all they know.

This lack of understanding isn't surprising. NTSC (National Television Standards Committee) and PAL (European & South American Television Standards) encoding are quite complex, and the reasons they distort your graphics are not always obvious. Many of these distortions were barely visible when running low-resolution games, but the next generation of consoles means that every game will be running at high resolution, so we can't ignore these problems any longer.

The three main problems with displaying computer graphics on a TV are interlaced flicker, illegal colors, and reduced resolution. This article will explain what these problems are, and why they exist. Most of the concentrate will be on NTSC, but many of the issues also show up in PAL.

This article will briefly explain the important differences between NTSC and PAL, and will suggest ways to create your content so that it will suffer less when being displayed on a TV.

You've spent thousands of hours making your models, drawing your textures and optimizing your code. Don't let ignorance of the TV encoder ruin your masterpiece.

Interlace

An image which is rock solid on a computer monitor may flicker quite badly when displayed on a television. The computer image appears solid because the complete image is scanned out to the monitor sixty times a second or more. This is fast enough that the phosphors flickering on and off are perceived by the human mind as continuous light. However, on a television monitor, each individual line is only scanned out thirty times a second. Since the "fusion frequency", where a flashing light seems continuous, is generally around 50-60 flashes per second (Hz) we see flickering (Taylor 01).

What's actually happening on a television is that the electron beam scans over the picture tube sixty times a second, but only draws half of the lines. That is called one field. The next scan of the electron beam draws the other lines. Those two fields make up one 'frame' of video. This means that while each line is only drawn thirty times a second, the screen as a whole is drawn sixty times a second. This is called an interlaced display. The field rate is 60 Hz and the frame rate is 30 Hz. This can be confusing since the video game meaning of 'frame rate' is slightly different from the video meaning; however, usually it's clear which is meant.

The reason that TVs behave this way is because the NTSC was trying to get the best picture possible into a 6 MHz channel, and 6 MHz isn't enough bandwidth to transmit a full 525 line frame sixty times a second. They had to choose the best way to deal with this limitation.

One option would have been to increase the bandwidth available per channel. If they had doubled it to 12 MHz per channel then they would have avoided interlaced flicker entirely. However, that also would have cut the number of channels in half, something that the viewing public was unlikely to accept.

Another option would have been to display 525 lines at 30 frames per second. However, this frequency is so far below the fusion frequency that the flicker would have been completely unacceptable, which nobody would have tolerated.

Another option would be to display half as many lines, 262, at 60 frames per second. This would have produced flicker free television, but the vertical resolution would have been halved.

It is worth noting that the movie industry deals with this same 'bandwidth' problem in a very different way. Since film is expensive, movies are shot at 24 frames per second. However, if they simply played them back at 24 frames per second the flicker would be unbearable. Instead, they display each frame twice, to give a 48 frame per second flicker rate. In a darkened theater, that's enough to seem continuous. It would seem that the same effect could be obtained by displaying each frame for a full twenty-fourth of a second, instead of flashing it twice, but in fact this interferes with the perception of movement.

The reason the television industry didn't adopt the same solution as the movie industry is simple, no frame buffers. Television is a real-time medium, with the phosphors on the television lit up as the information streams in. In 1941, there was no way of buffering up a frame and displaying it twice. This option has only become economically feasible very recently. The other reason that displaying a full frame buffer twice is not used, even with HDTV, is that 60 fields per second gives noticeably smoother motion than 30 frames per second.

Another tempting option that couldn't be used was phosphors that would glow longer after the electron beam left. This method would have introduced two problems. One problem would be that the sense of motion would actually be reduced, which is what occurs if you display a movie frame for a full twenty-fourth of a second. The other problem is that it is impossible to make phosphors that fade out instantly after precisely one thirtieth of a second, and the gradual decay of the brightness would produce substantial blurring of motion.

Therefore, as much as I dislike interlace, I have to agree that it was the right choice.

The good news is that the flicker isn't really that bad. Television shows and movies seem to flicker significantly less than computer graphics. If we can learn what the video professionals do right that we so often do wrong we can improve our results.

The bad news is that the trick is to avoid fine vertical detail. The worst-case flicker happens if you have a single-pixel high, horizontal white line on a dark background. In such a case, the fact that the monitor is being refreshed 60 times a second is irrelevant. Your only visible graphics are being refreshed 30 times a second, and that's nowhere near enough. You can also get strong flicker if you have a large bright area with a large dark area below it with a sudden transition, because on the transition area you are effectively displaying at 30 Hz.

The key is to avoid the sudden transitions. If you have a scan line of 50% gray between your bright white and your black then the flicker will be dramatically reduced, because when the white line is not being displayed, the gray line overlaps a bit and stops the image from fading away to black (Schmandt 87).

The best thing to do may be to vertically blur all of your static graphics including menu screens and some texture maps in order to soften sudden transitions in brightness. Alternately, most of the new generation of consoles will do this for you. As the image buffer is sent to the television they will carefully average together two to five adjacent scan lines to eliminate transitions that are too fast. While turning on an option to blur your graphics seems like an odd way to improve your image quality, even a desecration of your carefully drawn art, it really is the right thing to do. Maybe if we call it vertical filtering instead of vertical blurring it will sound more scientific and reasonable. Let's do that.

As an example, the Conexant CX25870/871 video encoder does five-line horizontal and vertical filtering. In order to reduce flicker without excessive blurring it adjusts the filter amounts on a pixel-by-pixel basis based on how flicker prone the imagery is (Conexant 00).

The reason that television programs don't flicker objectionably is because reality doesn't usually have high contrast, high-frequency information, and because some video cameras do vertical filtering. The sample height for each scan line in a video camera can be almost twice the height of a scan line (Schmandt 87), thus pulling in a bit of information from adjacent lines. Therefore, even if there is a thin horizontal line in a scene, it will be spread out enough to reduce the flicker. This is further justification for why we should do exactly the same thing with our graphics.

One important thing to understand about vertical filtering is that it is best done on a full frame. If you are running your rendering loop at 60 Hz and only rendering a half height image then vertical filtering will not work as well - the line that is adjacent in memory will be too far away in screen space. If vertical filtering is to work properly, it has to have the other field of data so that it can filter it in. This can be a problem on consoles where fill rate or frame buffer memory restrictions make it tempting to render just a half height image. If fill rate is the limiting factor then you can do field rendering at 60 Hz but keep the alternate field around and use it for vertical filtering. If frame buffer memory is the limiting factor, you may need to use a 16-bit frame buffer to make room for a full frame.

Just because you are rendering a full height frame doesn't mean that you have to display this frame at 30 frames per second. For most games the sense of motion is substantially improved at 60 frames per second, so you should render a full height frame, 60 times per second, then get the video hardware to filter it down to a single field on the way to the TV. This does mean that you have to render an awful lot of pixels per second, but that's what the beefed up graphics chips are for!

Giving up some of the 480 lines of resolution is hard to do. However, it can be helpful to realize that the reason the vertical filtering works is because adjacent scan lines overlap a bit. Therefore, you never really had 480 lines of resolution anyway.

The good news is that, for some purposes, you can simulate much higher vertical resolution. If you draw a gray line instead of a white line then it will be perceived as being thinner. This doesn't help if you want a lot of thin lines close together, but it is useful if you want a line that is half a pixel high, or a pixel and a half high.

The next time you're cursing the NTSC for inflicting interlace on you it's worth noting that when they designed the NTSC standard, the flicker wasn't as bad. The phosphors of the standard were much dimmer than the phosphors that are actually used today, and today's brighter phosphors, together with brighter viewing conditions, make us more flicker sensitive.

If you think making video games for an interlaced medium is annoying, you should try creating a video editing system. Video game authors have the luxury of synthesizing a full frame of graphics sixty times a second, something video-editing systems can't do. This makes for some interesting problems when you rotate live video on screen. The missing lines are suddenly needed and they have to be somehow extrapolated from adjacent fields.

If you are used to making video games that run at non-interlaced resolutions, like 320x240, then going to 640x480 does not mean that your vertical resolution has doubled. It has increased a bit, and that's good, but you need to understand from the beginning that if you create 480 lines of detail, it won't be visible. The actual vertical resolution is somewhere between 240 and 480, which means that small details necessarily cover around one and a half scan lines.

If it's any consolation, NTSC resolution could have been lower. It wasn't until the very last meeting of the first NTSC that the resolution was raised from 441 lines to 525 lines.

So remember, if your target console has an option to vertically filter your images, enable it, and render full frames. If it doesn't have such an option, do it yourself before putting the art in the game. There's no point in putting in beautiful vertical detail if it will be invisible and will give your customers a headache.