Detecting Memory Leak OnAIX.docx

1、Detecting Memory Leak On AIXMethods for Identifying Memory Leaks in AIX SystemsBy Barry J. Saad and Harold R. LeeApplication memory leaks can be difficult to detect and on modern multi-tasking systems where many applications are running, it can be even more difficult to identify the offending proces

2、s(es).This paper will explore the reasons for memory leaks, and using the AIX system performance tools, provide a methodology for identifying them to the process level. This paper will consist of four major parts: (1) definition and causes for a memory leak on a system, (2) examining a process heap

3、using svmon and ps, (3) a method for detecting memory leaks on a system, (4) a method for identifying individual processes which have memory leaks.The Causes for Memory LeaksThe Jargon File version 4.2.0 defines a memory leak as:memory leakn. An error in a programs dynamic-storeallocation logic that

4、 causes it to fail to reclaim discarded memory,leading to eventual collapse due to memory exhaustion. These problems were severe on older machines with small, fixed-size address spaces, and special leak detection tools were commonly written to root them out. With the advent of virtual memory, it is

5、unfortunately easier to be sloppy about wasting a bit of memory (although when you run out of memory on a VM machine, it means youve got a _real_ leak!).Memory is allocated to a processes heap by using the malloc() C library call or in C+ when a class is instantiated by calling a “constructor” which

6、 will “set up” the class by allocating the required memory for the class and initializing the members. When the memory is no longer required by the process, it is released for reuse by the free() C library system call or in C+ when the class is “torn down” by calling a “destructor” which will releas

7、e the memory for reuse. When a process terminates, all of the programs dynamic-store which is also called the processes heap and stack are returned to operating system for reallocation to other processes. Memory leaks occur when memory that is allocated to the heap is not released by the process and

8、 the heap will continue to grow. Over time, the process will either allocate all of the memory and fill its address space which may cause abnormal termination of the process or it will use up the virtual memory and cause the kernel to take protective measures to preserve the integrity of the operati

9、ng system by killing processes to relieve the virtual memory demands. Unfortunately, memory leaks are usually caused by long running processes and the algorithm which the kernel uses to reduce virtual memory demands is to start killing processes based on the “youngest” process (most recently created

10、) and working toward the “oldest”. This can have the effect of many processes being “killed” before the offending process is dealt with. Due to the complexity of programs and the levels of memory allocation and deallocation abstraction in object oriented programs, memory leaks are sometimes very dif

11、ficult if not impossible to completely eliminate. One approach to this problem is to limit the lifespan of a program where the program is terminated before any potential memory leaks are allowed to grow enough to become a problem. The Apache web server is an example of this approach. When the Apache

12、 server is started, a parent process started which allocates very little memory and does very little work. The parents job is to initially spawn some number of child server process set by the StartServers directive in the configuration file, and then to try to maintain the number of child server pro

13、cess between two levels either by creating or killing additional processes. This window is set by the MaxSpareServers and MinSpareServers directives in the startup configuration file. MaxRequestsPerChild directiveThe MaxRequestsPerChild directive sets the limit on the number of requests that an indi

14、vidual child server process will handle. After MaxRequestsPerChild requests, the child process will die. If MaxRequestsPerChild is 0, then the process will never expire.Setting MaxRequestsPerChild to a non-zero limit has two beneficial effects: it limits the amount of memory that process can consume

15、 by (accidental) memory leakage; by giving processes a finite lifetime, it helps reduce the number of processes when the server load reduces. Each child server processes will keep track of the number of requests which it will handle and after this number has been reached, the process will die and re

16、turn all of the virtual memory to the kernel heap for reallocation. The following snippet is from the configuration file instruction page. Notice the reason given for limiting the lifetime of the child:Not all applications have the flexibility of Apache to deal with potential leaks. Another strategy

17、 is for companies to schedule regular intervals for the applications to be recycled. This involves shutting down the applications and restarting them. While some reboot the entire system, this is not necessary unless there are mitigating circumstances such as a maintenance patch or a reboot specific

18、 tuning change (e.g. changing the asynchronous subsystem parameters).Another form of process memory leakage is called “heap fragmentation”. Heap fragmentation, which is the bane of operating system design, is something developers have had to contend with since the beginning. A google search on “heap

19、 fragmentation” will return over 64,000 hits illustrating the magnitude of this issue. Heap fragmentation is when available memory is broken into small, non-contiguous blocks. When this occurs, memory allocation can fail even though there is enough memory in the heap to satisfy the request, because

20、no one block of memory is large enough to satisfy the allocation request.For applications that have a low memory usage, the standard heap is adequate; allocations will not fail due to heap fragmentation. However, if the application allocates memory frequently and uses a variety of allocation sizes,

21、memory allocation can fail due to heap fragmentation.Heap fragmentation is caused by numerous subsequent malloc()s and free()s on the process heap with widely varying sizes. This cause “holes” to form in the contiguous heap and unless a malloc() is small enough to use the space within a “hole” the p

22、rocess heap must grow to satisfy the request. One strategy for dealing with heap fragmentation is to employ a technique called “Garbage Collection”. When a request cannot be satisfied from the heap due to fragmentation, the garbage collection routine is called which moves all of the “holes” to the e

23、nd of the heap, making all of the space available at the end of the heap. This garbage collection is analogous to defragmenting a hard drive, which moves all of the unused space to the end. Some of the early operating systems employed garbage collection techniques however this causes very erratic pe

24、rformance from the system since all activity had to stop while the memory was re arranged. The Java virtual machine (JVM), which is a “guest” operating system, currently uses this method with similar results. Garbage collection is not used on most operating systems today due in part to the erratic p

25、erformance it produces.Another strategy which some programs use to eliminate this is to manage the heap internally through the use of pointers. When the program starts there is a single memory allocation call to obtain the heap and then the program itself will allocate portions out of it to satisfy

26、the internal requests. Most data base programs like Oracle and DB2 use this method of internal memory management.Operating system developers continue to work on more efficient memory allocation algorithms and incorporate them into the kernels to reduce this issue to one on minimum impact. The Window

27、s developers have introduced a heap manager called the “Low Fragmentation Heap” (LFH) which places the holes in a Cartesian btree structure similar to the AIX 4.2 5.2 malloc routine known as the “Yorktown malloc” since it was developed by the Yorktown research facility. A new malloc called the “wats

28、on malloc” is available on AIX 5.3. The Watson malloc which is cache based and uses a simplified rbtree (red-black tree) structure increases malloc performance as well as reducing heap fragmentation. The new malloc() is optimized for large multi-threaded applications and time will tell how much more

29、 efficient it is. For more information on this malloc refer to the following link: Refer to the section - Understanding the Watson Allocation PolicyFrom a system administration perspective, heap fragmentation is dealt with the same manner as a memory leak. The applications must be quiesced and resta

30、rted.Examining A Process Heap Using svmon and psFor an overview of the virtual memory management at a process level, refer to the following articles and info pages:http:/w3-Article Understanding Virtual Memory ManagementAlso refer to the man or info page on the svmon and ps commandsThe processes vir

31、tual memory footprint consists of three separate items. The actual (1) heap and the (2) process stack which is addressed by (esid) register “2”, and the (3)shared library data which is addressed by register “f”. The loader segment “d” is shared by all processes and is ignored. # svmon -nrP 262188- P

32、id Command Inuse Pin Pgsp Virtual 64-bit Mthrd LPage 262188 mamain 65980 4784 0 65977 N N N Vsid Esid Type Description LPage Inuse Pin Pgsp Virtual 23e9 2 work process private - 53263 3 0 53263 Addr Range: 0.53247 : 65314.65535 1f09d d work loader segment - 3508 0 0 3508 Addr Range: 0.8684 63ad f work shared library data -