With Sony's PS2 console, you had about 32 MB of memory to fill with data. With Xbox 360, there is 16 times as much space to fill, but the hardware to do it is essentially the same standard DVD reader. Getting anywhere near optimal use out of the DVD is turning out to be more important than ever. In Just Cause we were faced with the task of filling a huge, 32x32km large game world with interesting content - without loading screens.
There are three major ways to read resources into memory.
Resources can be read at the game's or level's startup and kept in memory. This is best for resources used all the time, and critical resources that that we must guarantee are in memory at a specific time. Since these resources are read before the start of the game, we can optimize loading time by moving resources into this category at the cost of memory.
Data can also be read based on the camera's world position. As the camera moves, new resources are read and old resources are evicted from memory. This group is best for resources spread uniformly in the world, and resources that are kept inside an area or a zone.
Finally, data can be read based on some event, for example when the player talks to a NPC. When we load data based on an event we need to ensure we do not load data that directly affects the player, such as loading a physical item at the characters position. This has to be taken into consideration in the game design, as we can never guarantee the latency of the read.
When we designed our streaming system, the most important design criteria was to minimize seeks while keeping our memory budget. I recommend that you load all data at the initial game/level loading, if you can get away with it in your type of game.
Some consoles only have a DVD for reading data. How fast we can read data depends on data layout and the quality of the media. Every time we switch layers to read from, it will cost us about 100 ms. In practice, you want all streamed data on a single layer and use the second layer for in-game movies or other data that is not used frequently. Each seek will cost us about 100 ms, and a safe estimate for sustained data rate is 10 MB/s. During the time we do one seek, we could have read 1 MB of data instead. It is almost always a good idea to duplicate data if it helps to avoid seeks. If you are designing for Blu-Ray and PS3, you will need to adjust these estimates.
A good streaming system should be designed to always read data asynchronously, as nothing kills performance as blocking synchronous I/O. Asynchronous I/O can be implemented either by using asynchronous I/O functions like ReadFileEx() in Win32, having a separate thread call the I/O functions, or using a dedicated CPU as the IOP on the PS2. When using system calls for asynchronous I/O, note that these may be synchronous when reading from a development kit's hard drive. Always measure your actual read performance using burned discs or, if possible, a DVD emulator.
On Microsoft's system, asynchronous callbacks will not be processed until you call the SleepEx() function. A typical solution will call SleepEx() once per frame, which will cause a small amount of time not using the device between the completion of the I/O request and the call to the SleepEx() function. All of these small times quickly add up, especially when reading many small files.
The best approach here is probably to use a hardware thread to read data, which will work on all platforms and give good performance. The downside is that it makes the system harder to debug.
Using asynchronous I/O has some implications for level design. Game logic can never assume that a streamed resource is ready at the moment it is requested. For example, if a character is scripted to shout "charge" before attacking, the script has to wait for the resource to finish loading before actually attacking.