deconstruct object from pointer array might not release memory?

davidz001 (5)
I am having a strange C++ new/delete question which is at some circumstance I delete (deconstruct) an object from pointer array but the memory seems not to be released. Please see the code and comments below.

If run all the steps from Step-1 to Step-6, the memory is NOT released after Step-4 or Step-5.

If run Step-1, Step-2, Step-3, Step-4 and Step-6 (comment and skip Step-5), the memory is NOT released after Step-4 or Step-6.

If run Step-3, Step-4 and Step-6 (comment and skip Step-1, Step-2 and Step-5), the memory can be released as expected after Step-4 or Step-6.

If run all the steps from Step-1 to Step-6, but at Step-3 the new string size is larger than previous size at Step-1, the memory can be released as expected after Step-4 or Step-5 or Step-6.

So in general, if NOT run Step-1 and Step-2, everything seems fine. But If run Step-1 and Step-2 before, something wired happens that the memory is NOT released unless the new string size is larger than previous size at Step-1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
 
    #include <iostream>
    #include <list>
    #include <unistd.h>

    struct MyClass {
        std::string str;
        MyClass() {}
        ~MyClass() {}
    };

    int main(int argc, char* argv[]) {
        std::list<MyClass*> mylist;
 
        // Step-1: create 100 MyClass pointer array,
        // then construct 20 MyClass objects with 1MB's string for each
        // and put them into a list
        MyClass** pt1 = new MyClass*[100];
        for (int i = 0; i < 20; ++i) {
                std::string tmp_str(1024*1024, 'a'); // 1MB
                pt1[i] = new MyClass();
                pt1[i]->str = tmp_str;
                mylist.push_back(pt1[i]);
        }
        std::cout << "Step-1: creating done: " << mylist.size() << std::endl;
        sleep(10); // now check the memory usage of this process, it should use
                   // about 20MB memory


        // Step-2: delete all the MyClass objects from the list,
        // then delete pt1
        while (1) {
                std::list<MyClass*>::iterator it = mylist.begin();
                if (it == mylist.end())
                        break;
                delete *it;
                mylist.erase(it);
        }
        delete [] pt1;
        pt1 = NULL;
        std::cout << "Step-2: deleting done, left: " << mylist.size()
                << std::endl;
        sleep(10); // now check the memory usage (RSS) of this process, 
                   // it should reduce about 20MB memory 


        // Step-3: create another 100 MyClass pointer array,
        // then construct 10 MyClass objects with 1MB's string for each
        // and put them into a list
        MyClass** pt2 = new MyClass*[100];
        for (int i = 0; i < 10; ++i) {
                std::string tmp_str2(1024*1024, 'b');
                pt2[i] = new MyClass();
                pt2[i]->str = tmp_str2;
                mylist.push_back(pt2[i]);
        }
        std::cout << "Step-3: creating done: " << mylist.size() << std::endl;
        sleep(10); // now check the memory usage (RSS) of this process
                   // it should use about 10MB memory


        // Step-4: delete 4 MyClass objects from the list, NOT all of them
        int j = 0;
        while (1) {
                std::list<MyClass*>::iterator it = mylist.begin();
                if (it == mylist.end() || ++j == 5)
                        break;
                delete *it;
                mylist.erase(it);
        }
        std::cout << "Step-4: deleting done, left: " << mylist.size()
                << std::endl;
        sleep(10); // now check the memory usage (RSS) of this process,
                   // we expect it should reduce about 4MB memory,
                   // but it still uses about 10MB memory and seems
                   // no memory is freed.


        // Step-5: delete all the left MyClass objects from the list
        while (1) {
                std::list<MyClass*>::iterator it = mylist.begin();
                if (it == mylist.end())
                        break;
                delete *it;
                mylist.erase(it);
        }
        std::cout << "Step-5: deleting done, left: " << mylist.size()
                << std::endl;
        sleep(10); // now check the memory usage (RSS) of this process,
                   // we expect it should reduce about 10MB memory,
                   // but it still uses about 10MB memory and seems
                   // no memory is freed.


        // Step-6: delete pt2
        delete [] pt2;
        pt2 = NULL;
        std::cout << "Step-6: deleting array done" << std::endl;
        sleep(10); // now check the memory usage (RSS) of this process,
                   // if we run Step-5 before, then the memory will reduce
                   // about 10MB,
                   // but if we didn't run Step-5 before, then the memory will 
                   // still be about 10MB and seems no memory is freed. 
        return 0;
    }
Last edited on
Cubbi (4774)
Normally, we don't use new/delete in C++
But either way, the means by which you're measuring memory use are apparently not sufficiently precise. Running your program with valgrind, I get:

$ valgrind ./test
==30288== Memcheck, a memory error detector
==30288== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30288== Using Valgrind-3.12.0 and LibVEX; rerun with -h for copyright info
==30288== Command: ./test
==30288== 
Step-1: creating done: 20
Step-2: deleting done, left: 0
Step-3: creating done: 10
Step-4: deleting done, left: 6
Step-5: deleting done, left: 0
Step-6: deleting array done
==30288== 
==30288== HEAP SUMMARY:
==30288==     in use at exit: 0 bytes in 0 blocks
==30288==   total heap usage: 124 allocs, 124 frees, 62,991,628 bytes allocated
==30288== 
==30288== All heap blocks were freed -- no leaks are possible
==30288== 
==30288== For counts of detected and suppressed errors, rerun with: -v
==30288== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
davidz001 (5)
Thanks, Cubbi.

I know no leaks here and this code is an example, I just want to know why the memory is not released after Step-4?
davidz001 (5)
Hi Cubbi,

Actually I use boost::pool to manage the memory, but I met the same behavior. And boost::pool is using new/delete underlying, so I write this test code to verify the behavior:).
Cubbi (4774)
I just want to know why the memory is not released after Step-4?

Memory is released after Step-4 as valgrind shows. To try to answer why you're not observing it, we need to know exactly what you're using to observe it, on what OS (and libc version may be significant too)
Last edited on
davidz001 (5)
Hi Cubbi,

I use "ps auwx" to check this program's RSS when it is sleeping on Linux 3.10.102. My glibc version is 2.17.

Does free release the memory to C++ library? C++ library itself will maintain some memory blocks and decide when to release them back to OS?

Thanks.
JLBorges (13770)
> Does free release the memory to C++ library?
> C++ library itself will maintain some memory blocks and decide when to release them back to OS?

Yes. More information: http://stackoverflow.com/questions/5994543/problem-usage-memory-in-c
davidz001 (5)
Cubbi (4774)
I don't have exact same linux (the box I'm trying this on still has glibc 2.12), but I see this too and this is actually a good learning opportunity for the Linux malloc

basic info: Linux has two heap memory allocation strategies: sbrk and mmap.

sbrk (system break) is a single contiguous virtual memory range. malloc can ask the OS to grow or shrink sbrk (in one direction).

mmap (memory mapping) is a separate contiguous virtual memory range. malloc can ask the OS to create and destroy (not resize) multiple mmap regions

The glibc malloc uses both, choosing based on some heuristics

tools in use: strace, you can also call mallinfo() if you feel like digging deeper into sbrk contents or /proc/pid to follow the OS side of things

Before Step-1:

malloc asks for sbrk and grows it by 0x21000 (135k)
brk(0)                                  = 0x12c0000
brk(0x12e1000)                          = 0x12e1000

then malloc allocates 20 independent mmaps 1052672 bytes each
mmap(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f706b2b0000
...
mmap(NULL, 1052672, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7069167000
and one more mmap 4096 bytes in size
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f706b3ca000


After this you print "Step-1" and sleep

Before Step-2:

malloc releases the 20 mmaps to the OS - this is why ps output shows RSS reduced
munmap(0x7f706b2b0000, 1052672)         = 0
...
munmap(0x7f7069167000, 1052672)         = 0


After this you print "Step-2: deleting done"

Then malloc changes strategies and requests memory for your 10 objects by growing the system break 10 times by 0x101000 (1052672 bytes) each time

brk(0x13e2000)                          = 0x13e2000
brk(0x14e3000)                          = 0x14e3000
brk(0x15e4000)                          = 0x15e4000
brk(0x16e5000)                          = 0x16e5000
brk(0x17e6000)                          = 0x17e6000
brk(0x18e7000)                          = 0x18e7000
brk(0x19e8000)                          = 0x19e8000
brk(0x1ae9000)                          = 0x1ae9000
brk(0x1bea000)                          = 0x1bea000
brk(0x1ceb000)                          = 0x1ceb000


Here you print "Step-3: creating done:"

After that, malloc works within the system break, with no communication to the OS, as you print "Step-4: deleting done" and "Step-5: deleting done" and "Step-6: deleting array done"

You end up with an empty sbrk. It's great if you plan to malloc again, but not if you don't.

Linux has a function for that:

malloc_trim(size_t pad);
If possible, gives memory back to the system (via negative
arguments to sbrk) if there is unused memory at the `high' end of
the malloc pool. You can call this after freeing large blocks of
memory to potentially reduce the system-level memory requiremen


let's call it (note you can also make it happen automatically with mallopt()):

1
2
3
 
        pt2 = NULL;
        malloc_trim(0);
        std::cout << "Step-6: deleting array done" << std::endl;


now malloc makes a brk call before printing Step-6, undoing everything almost all the way back to the break before Step-1:
brk(0x12e1000)                          = 0x12e1000
brk(0x12c1000)                          = 0x12c1000

and RSS drops in my ps output


So, why did malloc decide to switch from mmap to sbrk? Linux is open source, looking through
https://github.com/lattera/glibc/blob/master/malloc/malloc.c I see 
1
2
3
 
 /* the mmap_threshold is dynamic, until the user sets
     it manually, at which point we need to disable any
     dynamic behavior. */


so your other option is to set that threshold by calling mallopt() (this is what disables automatic behavior).. or learn the logic and adapt to it. It being Linux, you can also load different mallocs.
Last edited on
Topic archived. No new replies allowed.