Location Independence and Format Flexibility

Your game engine (in non-shipping form) should be able to read data in a shipping/optimized form and many non-shipping/non-optimized forms. It also should be able to read it from different physical locations.

For example, optimized stream creation can be very expensive and require tons of source content and/or intermediate data to be available. It will always be better to be able to read the data without ever doing the stream creation.

Be able to retrieve records from several locations on the target and from remote locations, and have this be transparent to the upper layers of game code.

Load Only What You Need

Game engine load times are bad, especially during development. Viewing tools sometimes have bad load times too. Pay for the load once, and then load only the data that has changed.

In addition, to make it easier to "try things out", get a network connection going between the host and target, and set up a reflection mechanism so you can change data in running objects on the target without changing what's on the hard drive. This is great for particle system tuning, sound tuning, and in lots of other situations.

Keep the Data Moving

Always enforce a preference of memory over disk over network, but above all, measure!

For example, don't write data out to disk with one custom tool, only to read it back in with another custom tool -- when the two tools might have been architected to work together and act on the data entirely in memory.

Sometimes this is not possible because an external tool is not architected to allow the data to move in this manner. However, when you're building your own tools and are in control of your own destiny, make sure you don't "leave money on the table," so to speak, by overlooking these important opportunities to optimize.

Only Compress and Compile When You Must

Compression and compiling can be time-intensive functions. Assume that your target application can handle an uncompressed or uncompiled version.

If loading the raw form of data into the target application is shorter than the compression or compiling time, plus the time to load the compressed or compiled data, then you're better off not compressing or compiling in the first place. This goes hand in hand with format flexibility, as mentioned above.

Reduce Complex Dependencies

The more data you require to perform a transform on your machine, the more likely it is that you will have to retrieve or build that data to perform the transform.

This is sometimes unavoidable, but you should try to minimize it wherever possible. This comes into play particularly in data packing steps, where many records are packed into one big file. Avoid data packing steps during host-target iteration.

There's another way to look at the same thing, from a hard drive perspective:

Size of data written = Size of data actually related to change + Size of incidental data

There's no hard-and-fast rule, but if you have to write a ton of data to the hard disk, and only some of it is actually related to the change you made, you have an inefficient pipeline.