1

I have a mutlithreaded linux server (64 bit) which is supposed to run and handle the requests for days. But for some time now I am seeing sudden jump in the memory utilization of the process. Sometimes this comes down after some time but sometimes my process is crashed after it reaches the threshold limit.

I used the smaps and pmap to find out the mappings and found that the heap size was 390 MB whereas the total memory utilization at that time was 4.5GB.

I could see a lot of anon memory segments in the pmaps output so I ran the strace and found that at the time of memery jump my process was calling mmap with 134MB size:

29045 19:52:45 mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x2aabf8000000
29045 19:53:12 mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x2aac00000000
29045 19:53:21 mmap(0x2aac04000000, 67108864, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x2aac04000000
29045 19:53:28 mmap(NULL, 134217728, PROT_NONE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_NORESERVE, -1, 0) = 0x2aac08000000

I wrote my wrapper for malloc and found that the maximum memory allocation done by my application was One 30MB allocation and one 20MB allocation at the time of jump. And this memory was freed.

I need to know why my process is calling these big mmaps?

Additional Information:

I breaked at mmap and found following BT:

#0  0x00000032af6d0940 in mmap64 () from /lib64/libc.so.6
#1  0x00000032af66f9cf in new_heap () from /lib64/libc.so.6
#2  0x00000032af673515 in _int_malloc () from /lib64/libc.so.6
#3  0x00000032af674cde in malloc () from /lib64/libc.so.6

The malloc was called with 5060 bytes but mmap was called with size 134217728. Why malloc is calling new_heap()?

Mahendra Raghuwanshi
  • 11
  • 2
  • It might be that malloc() creates a new heap for "small" allocations. Depending on the malloc implementation you might get different heaps for different allocation sizes (eg. one for malloc() calls between 4096 and 8092 bytes, another one for allocations between 10 and 20MB...). Can you check the backtraces for the other mmap() calls and check their allocation sizes? – oliver Jul 29 '13 at 08:54
  • Also, if you do lots of small allocations, maybe your heap is getting fragmented, leading to wasted memory? Can you check how many malloc() and free() calls with small allocations you are doing? Also, see http://stackoverflow.com/questions/3770457/what-is-memory-fragmentation for an overview. – oliver Jul 29 '13 at 08:59

1 Answers1

0

You could try attaching GDB to the process and set a conditional breakpoint on mmap(). That would give you a backtrace for the huge allocations.

oliver
  • 5,451
  • 8
  • 41
  • 48
  • Ok I will try this, but my code does not call the mmap directly anywhere. Its surprising that my heap is not that big and malloc is not being called with big size so where else my memory is going? – Mahendra Raghuwanshi Jul 29 '13 at 06:50