Features

Monitoring Your Console's Memory Usage, Part One

PS2

The Playstation 2 does not have the Xbox's great debugging tools. It also lacks a unified memory structure. On the bright side, the heap system is so simple that we can easily write our own heapwalker.

Dumping the memory

Intercepting all allocations

On the PS2, there is no global allocation that can be intercepted as on the Xbox. Of course, the new operators can be overloaded, but there is no way to intercept any other allocation functions. That leaves us with just one option: wrap all allocations! Since we used Renderware Graphics for our game, we simply called their allocation routines. Renderware's allocation routines can be redirected, so we redirected them to our own custom allocation routines. Now all allocations were done through Renderware, and therefore, through our custom allocation functions.

Overloading new and delete operators is even easier, so after our game used just these functions, most of the memory was intercepted. With a few exceptions...

The Sony runtime libraries allocate memory from the heap as well. More specifically, printf and atof were the two functions bugging us. They allocated small memory blocks as soon as they needed them, causing fragmentation. We couldn't capture them because they used malloc_r directly. Malloc_r is an internal allocation routine from the runtime libraries. In the end we made sure that on startup of our application printf and atof were called a few times to be sure they allocated all the memory they needed. The following code did the trick for us, and caused no memory fragmentation during the game.

	float dummy = 0.0f; dummy = atof("0.2123412341234"); dummy = atof("0"); dummy = atof("1e+6"); printf("%0.3f\n", dummy);
	Listing 5. Getting rid of atof's and printf's fragmentation.

Now that this issue was solved, we could intercept all other allocations used in our game, and we could add our 16-byte additional data to store our callstack information.

First we tried to store it at the beginning of each block, and simply return an address 16 bytes further, but somehow the Renderware DMA handler did not like that idea, so we ended up putting our data at the end of the memory block, which exposed a small quirk:

When we perform “malloc(8)” we get a memory block of at least 8 bytes, but when retrieving the block's size using: malloc_usable_size () we receive 12 as its size, which means the block is actually 12 bytes in size.

So, when we allocate 16 bytes extra, we should not put our information at “address+8;”, but at “address + malloc_usable_size(addres) – 16;” because otherwise we will be unable to find it later in functions such as free, realloc, and in our heapwalker.

Again, we mark our data with a tag such as 0xCAFEBABE. As mentioned earlier, this is needed because the Sony libraries allocate some memory too, and by using this tag we can identify whether the memory block has callstack information or not.

Callstack tracing on a MIPS machine is far more complicated then on Intel-based machines. I suggest reading, “See MIPS run” [REF1], for more detailed information.

Keith Packard from the MIT X Consortium created a callstack tracer algorithm for MIPS processors. It can be found in the Sony Developer Newsgroups [REF6]. This one already contains some modifications for EE specific instructions, and works like a charm for us.

The layout of the PS2's memory heap is very easy to parse. It is simply a large block of contiguous memory. First we have to find out where the heap starts. We are using CodeWarrior to build our project, and the following code will likely be different for other compilers such as GCC or ProDG.

In CodeWarrior, there is a feature called Linker Configuration Files (LCF), which, amongst other things, can be used to specify the heap size.

The CodeWarrior linker defines some symbols that can be accessed in the code. Sony's default heap implementation uses these symbols too, and so we are able to find out exactly where the heap starts.

typedef int __attribute__ ((mode (TI))) heap_size_type __attribute__((aligned(16)));
extern heap_size_type _end;

Adding the above two lines of code to one of your files makes it possible to find the exact address where your executable data ends. This also seems to be the start of the heap.

Using the following code, we can walk from the start of the heap until the exact end of the heap. By doing so, it accesses every single block of memory (listing 6).

	void HeapWalk() { int currSize, i; int currCode, nextCode; int lastBlock = 0; int freeBlock = 0; int heapStart = (((int)&_end) + 0x10); int* currHeader = ((int*)ms_HeapStart)-1; int* nextHeader = NULL; do { currSize = (*currHeader) & 0xfffffff0; nextHeader = currHeader + (currSize>>2); currCode = (*nextHeader) & 0x0000000f; lastBlock = (currCode == 0x09); freeBlock = (currCode == 0x00); currHeader += (currSize>>2); } while (!lastBlock); }
	Listing 6. PS2 Heapwalker.

	Variable Description currSize Size in bytes of the current block of memory, and as you can see, this is always a multiple of 16 bytes. currCode Flags for the current block of memory. These flags are always stored in the next memory block. nextHeader Contains the address of the next memory block's header. lastBlock Boolean indicating that the current block is the last block on the heap. freeBlock Indicates wether this block is free memory or used memory.

Dumping the memory

By walking the heap, we can also figure out what its end address is. We already had the start address, so by using the start and end address, we can dump the entire heap to file. This supplies us the actual contents of the memory. We will walk the heap again offline in MemAnalyze, using a slightly modified version of the HeapWalk function from listing 6. Listing 7 shows how you can dump the entire heap to file.

	void dumpHeap() { int heapStart = (((int)&_end) + 0x10); int heapEnd = GetHeapEndByWalkingTheHeap(); int fd = sceOpen("host0:heap.bin", SCE_CREAT | SCE_WRONLY); if (fd >= 0) { sceWrite(fd, (void*)heapStart, heapEnd - heapStart); sceClose(fd); } }
	Listing 7. Dumping the entire PS2 heap.

What's next?

Now that we can dump the heap data from both platforms to file, it is time to take a look at the tool. In part two, I'll discuss the details on map file parsing, PDB parsing, and take a close look at how the Xbox image is loaded into memory. We will also see how the tool processes the data from the memory dump to come up with several interesting views.

[1] See MIPS run, by Dominic Sweetman. Morgan Kaufmann Publishers, 1999 [ISBN: 1558604103]

[2] Playing with the stack, by Chavdar Dimitrov.
http://www.codeproject.com/tips/stackdumper.asp#xx324128xx

[3] XDK documentation: chapter "Xbox kernel memory management"

[4] Rob Wyatt's explanation on fragmentation and caching on Xbox

Xbox newsgroups: news.xds.xbox.com

Search for:

Matt Benic
D3D_AllocContiguousMemory question
08/12/2002

[5] Xbox Memory Architecture and Performance, by Mike Abrash.

Available in the XDK documentation and on Microsoft website:
https://xds.xbox.com/BPProgInfo.asp?Page=content/prog_wp_memoryarch.htm

[6] Keith Packard's algorithm for callstack tracing on MIPS processors

Sony Developer Newsgroups (news.ps2-pro.com)

Search for:

Phil Camp (SN Systems) <phil@snsys.com>
sce.dev.prog.ee
Tuesday, February 04, 2003 2:22 PM
Re: call stack trace for EE?

[7] Metrowerks' CodeTEST
http://www.metrowerks.com/MW/Develop/AMC/CodeTEST/CodeTEST+Memory.htm

[8] Compuware Boundschecker
http://www.compuware.com/products/devpartner/bounds.htm

[9] Forrest Trepte's training session on Xbox memory management
https://xds.xbox.com/media/Memory%20Management_files/default.htm

join | contact us | advertise | write | my profile
news | features | companies | jobs | resumes | education | product guide | projects | store



Copyright © 2003 CMP Media LLC

privacy policy | terms of service