Program memory usage
There are several reasons why one might want to figure out how much memory a certain program requires. This article presents two methods for determining memory usage of a program in linux/unix compatible environments.
Checking program memory usage with pmap
The pmap program available on linux/unix platforms reports memory map of a running process. The pmap takes the process ID as argument. For example, the following command shows memory usage of a vim session :
pmap -x 7398
The PID 7398 was obtained using the ps command. The result of the pmap command above is the following :
7398: vim test 08048000 1556K r-x-- /usr/bin/vim.basic 081db000 24K rw--- [ anon ] b7fe2000 308K rw--- /usr/lib/libpython2.6.so.1.0 ... b803f000 4K rw--- /usr/lib/libgpm.so.2.0.0 b8094000 4K rw--- /lib/libncurses.so.5.7 b8099000 4K r---- /usr/lib/gconv/EUC-JP.so b80c3000 4K rw--- /lib/ld-2.9.so bfad5000 84K rw--- [ stack ] total 7704K
The last line starting with 'total' indicates that the bash program uses a total of 7704KB of memory. Note that the reported total memory usage is based on the current snapshot of the current memory map. The program may use more memory if at certain point it dynamically allocates memory.
Valgrind Massif heap profiler
The Valgrind tool suite provides a number of debugging and profiling tools that help make programs faster and more correct.
The Massif is the heap profiler tool of valgrind . It measures how much heap memory your program uses. This includes both the useful space, and the extra bytes allocated for book-keeping and alignment purposes.
Heap profiling is useful when you want to know the peak memory usage of a program, and when your program does not run for long enough to run the pmap command.
To use the valgrind massif heap profiler, you need to first run valgrind as follows :
valgrind --tool=massif --depth=1 prog prog-args
Where prog and prog-args are the program you want to profile and the corresponding arguments. The –depth=1 option is used to reduce the granularity of heap memory inspection, since we are not debugging memory usage and only interested in the total memory usage.
In our case, we run vim without arguments (valgrind –tool=massif vim), then perform some file editing then exiting.
This will produce a file massif.out.xxxx that we will use as input for the ms_print utility to obtain the actual memory usage report. This is done by invoking ms_print as follows :
ms_print massif.out.xxxx
The resulting output will be :
--------------------------------------------------------------------------------
Command: vim
Massif arguments: --depth=1
ms_print arguments: massif.out.7615
--------------------------------------------------------------------------------
KB
877.6^ # .. ...
| , @# :: :::
| @@ @# :: :::
| : @@ @# :: :::
| : @@ @# :: :::
| , : @@ @# :: :::
| . ..@ : @@ @# :: :::
| : ::@ : @@ @# :: :::
| :: ::@ : @@ @# :: ::: :::@:: :::@
| ,. .: :: ::@ : @@ @# :: ::: :::@:: :::@
| .@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| .: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| ..:@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| , ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| , : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| .: @ : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| . :: @ : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| :: :: @ : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| . : :: :: @ : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
| : : :: :: @ : @ ::::@ :: :::: :@@: :: :: ::@ : @@ @# :: ::: :::@:: :::@
0 +----------------------------------------------------------------------->Mi
0 122.3
Number of snapshots: 53
Detailed snapshots: [7, 9, 14, 22, 23, 31, 33, 34, 35, 36 (peak), 45, 51]
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
0 0 0 0 0 0
1 3,400,849 67,856 63,438 4,418 0
2 5,910,026 96,208 82,583 13,625 0
3 9,301,525 141,336 114,196 27,140 0
4 11,718,411 167,968 132,652 35,316 0
5 15,268,708 202,704 156,741 45,963 0
6 17,712,798 224,992 172,064 52,928 0
7 21,116,113 252,800 191,160 61,640 0
75.62% (191,160B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.68% (186,271B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.93% (4,889B) in 16 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
8 24,270,649 278,808 209,494 69,314 0
9 27,073,846 297,536 222,283 75,253 0
74.71% (222,283B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.06% (217,394B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.64% (4,889B) in 16 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
10 30,479,325 324,392 241,523 82,869 0
11 32,585,346 337,648 250,600 87,048 0
12 34,937,065 351,312 259,880 91,432 0
13 37,333,012 366,872 270,952 95,920 0
14 39,013,404 375,616 276,910 98,706 0
73.72% (276,910B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->72.42% (272,021B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.30% (4,889B) in 16 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
15 41,423,053 389,384 286,501 102,883 0
16 43,923,054 406,336 298,860 107,476 0
17 46,494,139 418,184 308,225 109,959 0
18 48,457,625 422,336 311,957 110,379 0
19 50,454,076 421,744 309,886 111,858 0
20 52,541,357 424,728 311,658 113,070 0
21 56,046,990 446,808 330,171 116,637 0
22 57,662,662 456,464 339,167 117,297 0
74.30% (339,167B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.07% (333,538B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.23% (5,629B) in 32 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
23 60,042,335 473,696 351,373 122,323 0
74.18% (351,373B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->72.91% (345,392B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.26% (5,981B) in 32 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
24 61,668,471 475,664 352,964 122,700 0
25 64,254,132 486,848 362,044 124,804 0
26 66,803,325 515,576 386,287 129,289 0
27 70,517,588 561,016 428,118 132,898 0
28 72,147,744 618,032 477,712 140,320 0
29 75,016,378 620,176 479,280 140,896 0
30 77,921,372 622,328 480,844 141,484 0
31 79,910,907 673,456 517,432 156,024 0
76.83% (517,432B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->75.09% (505,701B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.74% (11,731B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
32 82,985,534 771,784 598,516 173,268 0
33 86,854,317 817,640 639,179 178,461 0
78.17% (639,179B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->76.74% (627,448B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.43% (11,731B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
34 88,436,612 839,944 655,169 184,775 0
78.00% (655,169B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->76.60% (643,438B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.40% (11,731B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
35 90,886,012 864,912 677,205 187,707 0
78.30% (677,205B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->76.94% (665,474B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.36% (11,731B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
36 94,370,797 898,712 706,052 192,660 0
78.56% (706,052B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->77.22% (693,969B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->01.34% (12,083B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
37 96,321,715 876,696 683,477 193,219 0
38 98,274,530 878,720 685,221 193,499 0
39 101,835,680 888,944 695,722 193,222 0
40 103,881,055 889,016 695,768 193,248 0
41 106,092,033 888,952 695,732 193,220 0
42 109,171,195 548,280 412,961 135,319 0
43 111,133,276 550,656 414,848 135,808 0
44 113,190,282 550,376 414,645 135,731 0
45 115,145,024 557,120 420,834 136,286 0
75.54% (420,834B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.43% (409,103B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->02.11% (11,731B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
46 117,246,258 550,848 414,903 135,945 0
47 119,194,607 553,224 416,827 136,397 0
48 121,220,195 552,792 416,626 136,166 0
49 123,377,661 553,176 416,819 136,357 0
50 124,961,128 555,048 418,504 136,544 0
51 126,673,425 554,832 418,196 136,636 0
75.37% (418,196B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc.
->73.32% (406,817B) 0x81167D6: lalloc (in /usr/bin/vim.basic)
|
->02.05% (11,379B) in 40 places, all below massif's threshold (01.00%)
--------------------------------------------------------------------------------
n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B)
--------------------------------------------------------------------------------
52 128,256,865 556,632 419,776 136,856 0
Valgrind takes snapshots of the memory usage at each instruction and creates an entry if memory usage has changed.The graph indicated the memory usage through the execution of the program, we can see that vim used a maximum of 877.6KB at a certain time. Just under the graph, there is table with snapshot identifiers. The commentary (peak) indicates the snapshot that corresponds to maximum memory usage during the vim session.
| Labels: unix, coding |
|