Receiving "segmentation fault:11" after malloc()ing 12 GB of memory despite having enough memory available (32 GB)

Question

I have a program that needs to allocate 2 1.5billion-length integer arrays. It's for a coding challenge (https://projecteuler.net/problem=282) and there isn't a way around using such large arrays (if there is, please don't tell me; I'm supposed to find the answer on my own). They need to be 32-bit integers since their values are between 0 and 1.5 billion. 3 billion integers takes up about 12 gigabytes, so I decided to use an EC2 r5.xlarge instance with 32 gigabytes of memory, but I get a segmentation fault error in my C code. When I test the code locally, it works for smaller arrays but receives a segmentation fault:11 error on the full-length version.

I've looked online, and have tried changing settings in ulimit with ulimit -m 15000000 and ulimit -v 15000000 (both numbers are in kbytes). These were already set to unlimited so I don't think this did anything.

The C code

#include <stdio.h>
#include <stdlib.h>
#include <math.h>
int main(int argc, char* argv[]) {
    int magic = pow(14, 8); // 14**8 is 1,475,789,056
    // more lines
    int* a = malloc(4 * magic);
    int* b = malloc(4 * magic);
    if (a == NULL || b == NULL) {
    printf("malloc failed\n");
    exit(0);
    }
    for (int i = 0; i < magic; i++) a[i] = (2 * i + 3) % magic;
    // some more lines

I get a segmentation fault error in my C code on the EC2 instance. When I test the code locally, it prints out the correct value for smaller length arrays but receives a segmentation fault:11 error.

Just a suggestion, Euler problems are intended to be solved without having to brute force calculate. If you're having to allocate massive amounts of memory, you're taking the wrong approach to the problem. — blackbrandt, Jul 01 '19 at 19:52
@blackbrandt my solution is a much more efficient solution than brute force (brute force is much worse than exponential in this problem). There might be a way to not use this much memory though. — NTitterton, Jul 01 '19 at 19:54
@MrTux if my math is correct, 2^32 > 14^8, which means that it would be able to fit into a long. — blackbrandt, Jul 01 '19 at 19:56
you need to pass long long integers or your value overflows 32 bit as 6 billion. Then you have to use a 64 bit target as a 32 bit executable cannot handle more than 4Gb of memory — Jean-François Fabre, Jul 01 '19 at 19:56
Since it's signed int, the max is 2^31. But that should still be big enough. — Barmar, Jul 01 '19 at 19:58
and yes, project euler problems are designed to avoid big memory allocations or big cpu computations for the final solution. bruteforcing is okay to get a reference and tune your algorithm, but for the real thing, you can't — Jean-François Fabre, Jul 01 '19 at 19:58
Ah thank you @Jean-FrançoisFabre. I changed `4 * magic` to `long mem = 4 * magic; int* a = malloc(mem);` and added `-m64` to my gcc compile line, but am still getting the same segfault error. — NTitterton, Jul 01 '19 at 20:02
The requirement "give your answer mod 14^8" is a big clue. 1475789056 fits a typical `int` type and quite often, the modulo can be done during the computation (except for division). — Weather Vane, Jul 01 '19 at 20:04
one last note: it seems to be fixed with `long mem = 4 * (long) magic;` — NTitterton, Jul 01 '19 at 20:16
It would have been magically fixed by you doing the correct thing instead of hardcoding the supposed size of int: `malloc(magic * sizeof *a)` — Antti Haapala, Jul 01 '19 at 23:31

Ilmari Karonen · Accepted Answer · 2019-07-01T20:11:58.690

Your magic number equals 1,475,789,056, which fits in a signed 32-bit integer just fine.

However, 4 * magic is 5,903,156,224, which doesn't, and overflows!

Technically, signed integer overflow in C is undefined behavior, so anything could happen at that point. But what typically happens is that the value is simply truncated to 32 bits, giving 1,608,188,928, and this is the number of bytes you end up allocating for each array.

And then your for loop, which tries to write 1,475,789,056 four-byte integers into that buffer, runs off the end of the array and eventually segfaults (possibly after corrupting some other values in memory first).

To fix this, store your magic number in a 64-bit variable, or at least cast it to a 64-bit type before passing it to malloc(). I would recommend using size_t, since that's the type actually intended for storing array sizes, and also the type that malloc() is defined to take in.

Of course, in general there's no guarantee that size_t has more than 32 bits, either — but if it doesn't, then you're out of luck anyway, since you're using 32-bit pointers and won't ever be able to allocate more than 4 GiB of memory (and probably not even that) in a single process, no matter how much total RAM you may have.

Thank you, I've changed what I'm passing into `malloc()` into a long and it now works. — NTitterton, Jul 01 '19 at 20:17

Receiving "segmentation fault:11" after malloc()ing 12 GB of memory despite having enough memory available (32 GB)

1 Answers1