17

If malloc/free is implemented as a library routine in libc, then is it implemented on top of the sbrk syscall or the mmap syscall, or something else?

And to be general, does the function declared in sys/syscall.h contains ALL the system calls in the target machine?

Pwn
  • 3,051
  • 11
  • 35
  • 42

3 Answers3

41

Very often, malloc and free are using lower-level virtual memory allocation services and allocating several pages (or even megabytes) at once, using system calls like mmap and munmap (and perhaps sbrk). Often malloc prefers to reuse previously freed memory space when relevant. Most malloc implementations use various and different strategies for "large" and "small" allocations, etc...

Notice that virtual address space can be limited, e.g. with setrlimit(2). Use on Linux pmap(1) and proc(5) to learn more about the virtual address space of some process (e.g. /proc/self/maps for your own one or /proc/1234/maps - also the pmap 1234 command - for process of pid 1234).

You could look at your GNU libc source code, look into the source code of other C standard libraries (such as musl-libc), read about malloc implementations, choose some other ones or implement your own, or use strace to find out experimentally.

Read the syscalls man page (i.e. syscalls(2)) and the file <asm/unistd.h> for a list of system calls.

a very fast malloc

(I believe that this could be the fastest implementation of malloc; however it is not very useful; it is conforming to the standards, e.g. n1570 or better)

I strongly believe that the C standard is very vague about malloc and free. I'm pretty sure that the following functions are respecting the letter (but not the spirit) of the standard:

 /* politically incorrect, but very probably standard conforming */
 void *malloc (size_t sz) { if (sz>0) errno = ENOMEM; return NULL; }
 void free(void*ptr) { }

Of course you'll code calloc and realloc accordingly.

(BTW every code using malloc should test against its failure, but some -incorrectly- don't; malloc can return NULL on failure and people should test against that case)

The GNU libc gives you hooks for your own malloc functions (and you could even probably use Boehm's Garbage Collector transparently thru them). These hooks could become deprecated and are non-standard.

If using GNU libc, look also into mallinfo(3) and malloc_stat(3) and related functions.

Basile Starynkevitch
  • 1
  • 16
  • 251
  • 479
  • 4
    Actually, `malloc` and `free` can be even simpler, something like `#define malloc(x) NULL` and `#define free(x) (void)1` :-) ISO doesn't require `errno` to be set, that's a POSIX thing. You'd probably also want to intercept `calloc` and `realloc` as well. – paxdiablo Jul 30 '18 at 04:39
  • 3
    @paxdiablo:are you sure that `malloc` can be just a macro? I thought it should be a function (assignable to a function pointer) – Basile Starynkevitch Jul 30 '18 at 05:51
  • 1
    Good point, I think you're right, I didn't think of that. You can still ditch the errno settings. – paxdiablo Jul 30 '18 at 07:53
13

malloc and free are standard C library functions which are to be implemented by each C implementation.

The C standard only defines the way in which these functions behave and the behavior expected from them. How they are to be implemented in left to each implementation.

In short they are implementation detail of the implementation you use.

(An "implementation" consists of the compiler, the linker, the runtime library, and probably a few other things.)

Keith Thompson
  • 230,326
  • 38
  • 368
  • 578
Alok Save
  • 190,255
  • 43
  • 403
  • 518
  • implemented by the platform's c library, not compiler. – u0b34a0f6ae Dec 10 '11 at 15:05
  • @kaizer.se: Compiler are free (heh!) to have their own standard library if they want, but there generally isn't much point. – dmckee --- ex-moderator kitten Dec 10 '11 at 15:27
  • 2
    @kaizer.se: It depends,A compiler can use the standard library implementation provided by the platform or they can provide their own implementation of the same.Which one to choose depends on the compiler.That is the meaning of *Implementation Detail* – Alok Save Dec 10 '11 at 17:24
  • 3
    Most commonly, `malloc` and `free` are implemented as library functions that invoke lower-level code. If an OS happens to provide system calls that exactly match the behavior that the C standard requires for `malloc` and `free, then they could be implemented as system calls. But I don't know of any OS that does this (even Unix, the birthplace of C). – Keith Thompson Dec 10 '11 at 23:37
3

You can also use an alternate implementation for malloc and free if you use a different memory allocator. For example, the hoard memory allocator is sometimes used to improve performance of multithreaded applications.

Peter Mortensen
  • 28,342
  • 21
  • 95
  • 123
Gabriel Southern
  • 8,316
  • 11
  • 50
  • 91