Understand the behavior of the new-handler

Question

I'm reading Effective C++ 55 by Scott Meyers and have a question from item 49:

When operator new is unable to fulfill a memory request, it calls the new-handler function repeatedly until it can find enough memory.

A well-designed newhandler function must do one of the following:

• Make more memory available.
• Install a different new-handler.
• Deinstall the new-handler
• Throw an exception
• Not return

When new can't allocate memory, it means that there is not enough memory, and the question is how and from where can the newhandler allocate more memory?

Can you explain all of these steps?

Answer 1

It depends on the implementation. I can tell you the way I usually do it:

1) The new handler allocates a large amount of memory at startup as a reserve.

2) When ordinary allocations fail, the new handler dips into its reserve.

3) Code that controls load management can hook the memory management system and determine when it has dipped into its reserve. It generally reacts by trimming caches and shedding load.

4) The memory manager attempts to refill its reserves as memory is freed.

5) When the reserve is restored, hooks are notified that they may grow caches and/or resume accepting additional load.

6) When the reserves get low, allocations that are able to fail (typically large allocations) fail. All code must sanely handle the failure of large allocations.

7) If the reserves are exhausted, allocations that are unable to fail (typically small allocations) block.

8) If the blocking condition persists or large allocations continue to fail and the reserve can't be restored, abnormal termination is triggered.

Answer 2

It can discard data that is not really needed. Say, photoshop caches the display image at several scales, and keeps as much as it can. This is how it knows just how much it can get away with.

Answer 3

Not return

It simply tried all it could and failed to make more memory available. It calls exit() or abort() and terminate the process.

This can be used as a fail fast strategy. If the memory is low on a system and it is known that some processes are not required (e.g.: child workers) calling abort() will free up memory for other processes and is the fastest way to get rid of processes.

throw an exception

Throwing bad_alloc means that the allocator cannot free any memory and if the application can somehow recover by. I personally don't care about actually catching bad_alloc as it should not happen or I cannot do anything about it.

This could be used if you are using ancient compilers that return nullptr when going out of memory to force them to throw. It is questionable whether this would work though.

Deinstall the new handler

Looking at the book calling set_new_handler(nullptr); should deinstall the new handler. According to Scott Meyer it should lead to operator new throwing the bad_alloc exception. This should be identical to throwing the exception yourself.

Install a different new handler

Since the new_handler is called until the allocation error is resolved (i.e.: it isn't resolved by operator new) if an implementation provided new_handler is not able to resolve the issue, it should propagate the condition to another layer.

If I would install a new_handler myself, I would keep a reference (see also: get_new_handler() )to the previous new_handler in order to swap it back in when my algorithm is failing.

Make more memory available

The assumption here is that the application installed the new_handler itself. The author of the new_handler has a chance here to introspect or modify the allocated memory. So the programmer has installed the new_handler because he knows there is some memory to free. In case everything goes wrong there is a strategy to recover from this failure.

An incomplete list of what might be going on (partial guessing):

• your operating system allows you to call the out of memory killer manually (to free memory)
• If you are using the Boehm garbage collector, you could actually call GC_gcollect(); to force garbage collection.
• you have reserved some memory yourself that you start using now.
• you manually call delete to known expensiable unimportant objects (caches).
• you have an interpreter running like Perl, Python or Mono and call its garbage collector to tell them you are in a low on memory condition (or they install the new handler themselves on their init() routines).

Implementation

Here is the implementation of libstdc++.