2

The question is regarding the allocation of arrays in .net. i have a sample program below in which maximum array i can get is to length. I increase length to +1 it gives outofMemory exception. But If I keep the length and remove the comments I am able to allocation 2 different big arrays. Both arrays are less the .net permissible object size of 2 gb and total memory also less than virtual Memory. Can someone put any thoughts?

 
class Program
    {
        static int length = 203423225;
        static double[] d = new double[length];
        //static int[] i = new int[15000000];
        static void Main(string[] args)
        {

            Console.WriteLine((sizeof(double)*(double)length)/(1024*1024));

            Console.WriteLine(d.Length);
            //Console.WriteLine(i.Length);
            Console.WriteLine(Process.GetCurrentProcess().VirtualMemorySize64.ToString());
        }
    }
Guy Coder
  • 22,011
  • 6
  • 54
  • 113
user1977781
  • 81
  • 1
  • 5
  • Similar to http://stackoverflow.com/questions/827015/max-size-of-net-arrays – RQDQ Feb 12 '13 at 13:11
  • What did you expect? The maximum array size depends on system conditions, and you keep changing the conditions. – Raymond Chen Feb 12 '13 at 13:13
  • If it's compiled for 32bit, in .NET you tend to get out of memory errors around 1.2-1.6GB; seeing as you're creating around 1.55GB of doubles, this is not surprising. See this for more info: http://stackoverflow.com/questions/1087982/single-objects-still-limited-to-2-gb-in-size-in-clr-4-0 – Chris Sinclair Feb 12 '13 at 13:15
  • You are trying to allocate more memory then is allowed. An integer is 32-bits yet you are looking at the 64-bit Virtual Memory Size your code makes no sense. – Security Hound Feb 12 '13 at 14:42
  • So is there a real world problem that you're trying to solve or are you just curious about memory management? Either is fine - just curious if there is more context here. – RQDQ Feb 12 '13 at 16:55
  • something similar at workplace was getting MemoryException. – user1977781 Feb 13 '13 at 06:17

2 Answers2

8

A 32-bit process must allocate virtual memory for the array from the address space it has available. by default 2 gigabytes. Which contains a mix of both code and data. Allocations are made from the holes between the existing allocations.

Such allocations always fail not because there is no more virtual memory left, they fail because the available holes are not big enough. And you asking for a big hole, getting 1.6 jiggabytes is very rare and will only work on very simple programs that don't load any additional DLLs. A poorly based DLL is a good way to cut a large hole in two, drastically reducing the odds of such an allocation succeeding. The more typical works-on-the-first-try allocation is around 650 megabytes. The second allocation didn't fail because there was another hole available. The odds go down considerably after a program has been running for a while and the address space got fragmented. A 90 MB allocation can fail.

You can get insight in how the virtual memory address space is carved up for a program with the SysInternals' VMMap utility.

A simple workaround is to set the EXE project's Platform target setting to AnyCPU and run the program on a 64-bit operating system. It will have gobs of addressable virtual memory space available, you'll only be limited by the maximum allowed size of the paging file and the .NET 2 gigabyte object size limit. A limitation that's addressed in .NET 4.5 with the new <gcAllowVeryLargeObjects> config element. Even a 32-bit program can take advantage of the available 4 gigabyte 32-bit address space on a 64-bit operating system with the /LARGEADDRESSAWARE option of editbin.exe, you'll have to run it in a post-build event.

Hans Passant
  • 873,011
  • 131
  • 1,552
  • 2,371
0

This will be because when allocating memory for arrays the memory must be contiguous (i.e. the array must be allocated as one large block of memory). Even if there is enough space in total to allocate the array, if the free address space is split up then the memory allocation will still fail unless the largest of those free spaces is large enough for the entire array.

Justin
  • 80,106
  • 47
  • 208
  • 350