Memory leak.

[blackcoder41]

How do you understand memory leak? Or what is memory leak for you?

I was thinking why would we manually free memory if we only use a few bytes, the operating system would claim it anyway after the program execution (I could be wrong by that)

[chrisname]

A memory leak is when you allocate memory, but lose the pointer (if it goes out of scope or you use it again) before you free it.

I had one in a virtual machine program with an SDL GUI. It was supposed to allocate an SDL_Surface to store the virtual machine's framebuffer (which programs running inside the vm would draw to) and then give the surface to a linked list, but as the function was called after every instrsuction (to make sure the screen was up to date with the framebuffer) it ended up jumping from a few hundred kiB of memory to several GiB in a few seconds.

int sint_gui_draw(char* buffer, long sz)
{
	SDL_Surface* surf = NULL;
	surf = SDL_CreateRGBSurface(SDL_SWSURFACE | SDL_DOUBLEBUF,
		  screen->w, screen->h, screen->pitch, 255, 255, 255, 255);
	surf->pixels = buffer;
	SDL_BlitSurface(surf, NULL, screen, NULL);
	SDL_Flip(screen);

	add_surface(surf);

	return 0;
}

There was a free_all_surfaces() function, though.

I took out the linked list because I realised that there was no need to store surfaces.

blackcoder41 wrote:

I was thinking why would we manually free memory if we only use a few bytes, the operating system would claim it anyway after the program execution (I could be wrong by that)

Yes, the heap gets freed once the program exits, but
1. It is good practice to free memory as soon as you're finished with it (even if you allocate it, do one thing with it, and then quit, I would still put a free() at the end of main())
2. If you don't free memory, you end up with the problem outlined above.

[z05DSL3A]

Or what is memory leak for you?

It is an unwanted increase in memory usage that is not/unable to be given back to the operating system during the execution of the application.

Frequent dynamic memory usage can result in fragmentation, thus an unwanted increase in memory usage but I'm not sure if everyone would consider that a leak, I don't think that I do.

[blackcoder41]

Grey Wolf wrote:

I'm not sure if everyone would consider that a leak, I don't think that I do.

Do you have any reasons why you don't? Just curious.

[z05DSL3A]

...because it doesn't quite meet my definition...well I guess it meets the definition above, maybe that is a bit brief and should include a clause that says that leaks are caused by programer mismanagement of memory management.

[rapidcoder]

There are also logical memory leaks - you allocate memory, you keep a pointer to it, but you never actually use it.

Frequent dynamic memory usage can result in fragmentation, thus an unwanted increase in memory usage but I'm not sure if everyone would consider that a leak, I don't think that I do.

I would classify that as memory fragmentation problems. The difference is that memory increase caused by fragmentation usually converges to some level, while in memory leak - it does not. If you run application with a memory leak for enough long, it will consume all the memory (or will get killed by the OS beforehand). However, in both cases the ratio of actively used memory to memory registered in the system may be huge (like 1000x or more in extreme cases).

Fragmentation is bad if your application allocates lots of small objects and then releases most of them, but not all of them. Then most of that released memory would not be given back to the OS. Remember that a 16B object may keep a whole, almost "empty", 4 kB memory page from being released.

[blackcoder41]

Anyone got a solution to fragmentation? I was thinking about allocating a dynamic array and reuse that through the life of the application. Well that is just my simple minded solution.

[chrisname]

I wouldn't go that far. I would just be wary of what you allocate on the heap. Put small variables on the stack wherever possible and free all of your heap data as soon as you're finished with it.

[rapidcoder]

Anyone got a solution to fragmentation?

Managed platforms with builtin GC got it - they move objects in memory to defragment the heap. But in native C++ you can write your own custom allocator that can allocate from different memory pools and carefully allocate objects by placement new. You are still constrained by impossibility to move objects in memory, though.

[Kyon]

Has anyone really been far even as decided to use even go want to do look more like?

chrisname wrote:

I wouldn't go that far.

[coder777]

Kyon wrote:

Has anyone really been far even as decided to use even go want to do look more like?

that's exactly what I wanted to say:))

Memory leak has another aspect besides the fragmentation and the loss of memory: There's simply something wrong with your code. which has the tendency to bring up more and more trouble. which actually brings you down ;)

But seriously: the cleaner your code is the faster are you to finish your task

[kbw]

blackcoder1

I was thinking why would we manually free memory if we only use a few bytes

I can think of a few reasons.
1. What if your leaky function gets called all over the place? Have you ever run a profiler on a C++ program and seen how many times a string constructor is called? Even a short run of a medium size program can rack up a few million in a minute or two.
2. Where does it end? Can we just forget to delete some little object over there too?
3. Some maintenance programmer comes along and sees your leak. The conclusion is that you may not know what you're doing. How many other leaks are there? It's just a waste of everyone's time not to do it correctly to begin with.
4. If you really want to not delete memory, do it properly. Use a smart pointer or a garbage collector. Although I don't think using a garbage collector is a proper solution to any problem.

blackcoder1

I was thinking about allocating a dynamic array and reuse that through the life of the application.

I have done this in the long and distant past, on a GIS system on MS-DOS.

The app required 512K memory, which was about all the memory there was for a DOS app. Once the app got going, the heap was so fragmented that we were almost certainly not going to get those big buffers that the app needed to do boundary calculations. Allocating the two large buffers at the start was sufficient, and of course they could be suballocated from in places when they weren't used.

[rapidcoder]

Although I don't think using a garbage collector is a proper solution to any problem.

It is a solution to the problem of manual memory allocation issues breaking your abstractions. E.g. thanks to GC you can make memory management totally independent from the API that you provide. There is lots of fuss about so called object ownership, but hard things become sometimes extremely simple when you drop the whole concept of ownership and say that objects "simply are", not "belong".

[kbw]

It is a solution to the problem of manual memory allocation issues breaking your abstractions.

That's new to me, can you please elaborate?

[rapidcoder]

Can you create a general-purpose caching library that could cache arbitrary object references (no copying)? In GCed languages it is a piece of cake. And you can make it in a way, that cache doesn't depend on the cached objects and vice versa. In C++ it impossible without introducing artificial dependency between the cache and the cached object classes - objects need to notify the cache before they are deleted. Otherwise, the cache would be unsafe and might return dangling pointers.

Introducing artificial dependencied between classes just to get memory management right is breaking abstractions (because something that is not related in the problem domain becomes related in the code). This is what I meant by breaking abstractions. Forcing the API designer to think of memory management issues, which are implementation detail, is breaking abstractions, too (in this case: encapsulation).

[z05DSL3A]

Forcing the API designer to think...

[sarcasm] Heaven forbid the designer should be forced to think. [/sarcasm]

[kbw]

That doesn't make much sense to me.

It assumes that there's only one memory. You don't need to care about where the object was allocated because it can only come from one place. C++ is more flexible than this as I'm sure you know. You're framework can present a more restricted view of the world, but that's not a constraint of the language.

You also assume that the asynchronous destruction of objects is fine. When operating in a GC environment you can't rely on destructors to clean up for you. You have to do it yourself, which defeats the point of destructors, right?

You also don't seem to care about the lifetime of objects. In GC'd languages objects are created and eventually released, but again due to the asynchronous nature of destruction, no one really knows when that is.

But for me, the real folly is that memory isn't the only resource. You have file handles, sockets, ... all sorts of things. And is it ok for them to be asynchronously destroyed?

I think my rant here sums it up for me:
http://slashdot.org/journal/235955/The-Folly-of-Garbage-Collection

[rapidcoder]

But for me, the real folly is that memory isn't the only resource. You have file handles, sockets, ... all sorts of things

Very old line of argumentation, and debunked many times in countless GC vs manual MM rants.
Memory isn't the only resource, but it is the resource that is used most of time. I am sure more than 99.999% of my objects are memory-only resources. :P And unlike many other resources memory is: stateful and shared. Which is the worst combination that can be. Just look at the topics on this forum - most of the problems related to resource management are memory-related problems, not e.g. file-related problems. When I read from file, I just open it somewhere, read the data into memory and close it near the place that has opened it. So the problem of managing such resources is well isolated to a small piece of code. The rest of the code doesn't operate on the file descriptor directly - it operates on data read from that file into the memory.
And in GCed environments you can have RAII-like behaviour too - in some of them it is not any more complex to achieve than in C++ (if you can't do it it is a problem of your crappy GCed language, not GC itself).

Heaven forbid the designer should be forced to think.

The design of proper interfaces is hard enough that making it even more complex can sometimes be just "too complex". And you know what happens when people deal with APIs that are too complex? Bugs, bugs, bugs... Interface should be only an interface - it should be as much independent from the underlying implementation as possible. GC allows to do it better. You asked what GC is a solution to. I answered. Just it. It simplifies modular design. Things like Eclipse where you have enormous number of plugins, that provide sometimes core functionality, would be extreme pain to make in a non GCed environment.

BTW: Do you think adding shared_ptr, a poor-man's GC, to Boost was pointless? Or people like H. Boehm wasted their time creating things like BDW? (ok, it is still a poor-man's GC compared to platforms designed with GC from ground up, but whatever, it is slow, but it works).

[sohguanh]

I believe the invention or should I say the implementation of GC in a lot of programming languages do help to cut down a lot of program bugs and crashes due to developer needing to do low level memory management.

With the advancement of modern hardware, I believe GC is ready to enter the stage for business application as witnessed by the sheer number of Java-based systems. However GC is not the holy grail. It does exact performance penalty and depending on business requirements, the small little time GC need to do it's work cannot even be tolerated at all. I would think of Stock Exchange trading software handling millions of transactions everyday as an example.

Hence, GC is here to stay but it cannot replace ALL other forms when their systems performance expectation cannot be met and handled. Would we be able to see the time when ALL systems are using GC as part of their systems underlying ?

Hmmm.. food for my thought today :O

[kbw]

When I read from file, I just open it somewhere, read the data into memory and close it near the place that has opened it.

Why not just release the object at this stage and let it release the resources?

The whole point of C with Classes was to guarantee initialisation and release of resources as it is always a common mistake that programmers make. The reliance on GC removes this benefit.

Do you think adding shared_ptr, a poor-man's GC, to Boost was pointless?

I don't think boost::shared_ptr is a poor-man's GC. The life time of an object managed by boost::shared_ptr is known and synchronous. It's not a GC in any sense of the word.

And in GCed environments you can have RAII-like behaviour too - in some of them it is not any more complex to achieve than in C++ (if you can't do it it is a problem of your crappy GCed language, not GC itself).

Now that sounds promising. Do you have an example?