The Growth of VRAM

Another factor in the PC memory equation has been growing as well: video memory size. In the early days of Direct3D, the typical video card had 16 or 32 MB of Video RAM (VRAM). High-end video cards now have 512 MB, 640 MB, 768 MB, or more VRAM. When video cards had 16 or 32 MB of Video RAM, this memory was mapped directly into every process that used Direct3D for efficient access by the application and video driver.

As video cards grew larger, this became unsustainable. A 768 MB hole in the 2-GB virtual address space of each process would leave very little space for applications. Similarly, taking 768 MB out of the 4 GB physical address space would be too constraining. This problem is exacerbated in dual GPU configurations (SLI®/Crossfire™).

Therefore, video card manufacturers typically implement a 256 MB physical memory window for the video graphics memory, and modern drivers do not create direct process mappings for the entire VRAM size. Process address space is still consumed for working with the AGP aperture (64 MB, 128 MB, or more typically on modern game systems 256 MB in size). While PCIe uses a dynamic aperture, it too is mapped into each process that uses Direct3D.

Beyond the direct impact of growing VRAM sizes, more process memory is needed to maintain the backing-store for handling “lost-device” situations so for textures, geometry, and other static data, filling up such large video cards and still fitting under the 2 GB limit is extremely challenging.

The Windows Vista Display Driver Model (WDDM) was designed to address the lost device limitations inherent to the Windows XP Display Model (XPDM), allowing more efficient sharing of the GPU by multiple applications. WDDM does not require the entire 256 MB aperture be mapped in to the process space.

Instead, it dynamically grows the amount as the VRAM allocated by the application increases. For Direct3D 10 applications, this eliminates the need to maintain two copies of resources like textures in memory, one for the VRAM and one in the backing-store for lost-device cases. The system deals with migrating the one copy between memory and the video card, as needed.

Unfortunately, to maintain application compatibility with Direct3D 9 running on Windows XP, two copies of managed resources were still maintained for Direct3D 9 applications running on Windows Vista. This also required process space for maintaining resources like render frame buffers. Previously, render frame buffers were simply lost and recreated. Therefore, they did not require any process space under WDM.

Since the elimination of the 256 MB aperture returned 256 MB of virtual address space to applications on WDDM that was already budgeted for under WDM, this change did not cause any problems until video cards with more than 256 MB of VRAM became available. Small games still had plenty of that 2 GB address space available, but many modern AAA PC titles were running out of space on WDDM.⁷ The WDDM VA hotfix (KB940105 for Windows Vista, included in Service Pack 1) ⁸ gives a bit of breathing room by only mapping video resources into the process that need direct CPU access. Games that use Direct3D 10 also have decreased memory pressure without the extra copies required for “device lost” handling.

However, high-end games in development are routinely hitting the 2 GB wall even on Windows XP. In fact, this incident proves that many modern AAA PC titles are already within 256 MB of the 2 GB barrier. Otherwise, they would not have hit this problem until video cards were over 512 MB.