How am I getting a "java.lang.OutOfMemoryError: Java heap space" if my heap isn't full?

Question

I'm debugging a fairly large project I've been working on (but did not originally create) and I've noticed that sometimes it crashes with an OutOfMemoryError. The code is loading a lot of data from files so this isn't entirely surprising in general.

However, what confuses me is that I'm using VisualVM 1.3.4 to profile the program, and it behaves inconsistently. Most times I've run it, the heap gradually expands up to about 2GB (the computer has 16GB of RAM; it's for academic research) with the used heap spiking higher and higher underneath it. Around 2GB, it will crash. The program isn't proccessing more information as time goes on though, so it shouldn't grow the heap to 2GB in just a few minutes.

Sometimes, though, I get a sudden crash after about 30 seconds, with a heap size of 250MB and only about 100MB in use. How am I getting a java.lang.OutOfMemoryError: Java heap space if my heap isn't full?

Edit: I'm using Eclipse to run the program, and I have the VisualVM plugin so it gets launched automatically. Also, I'm using Java 7.

What is the max heap of the JVM set to? It doesn't size to your RAM automatically. — Michael, Aug 09 '12 at 13:44
What version of Java are you using? What command line parameters are you using? (Some parameters will prevent your heap from resizing to use all of it) — Peter Lawrey, Aug 09 '12 at 13:46
Rules for the default heap size are here: http://stackoverflow.com/questions/4667483/how-is-the-default-java-heap-size-determined Furthermore, what are you using to measure the heap size... visual vm or the OS? Only the former is accurate regarding the heap. — Dan Gravell, Aug 09 '12 at 13:47
Do you bear in mind that JVM uses some space of your heap to works? I'm not saying that this is the general problem, but I bet that it's in the case of the heap size=250MB — Charliemops, Aug 09 '12 at 13:50

score 2 · Answer 1 · answered Aug 09 '12 at 13:48

2

Start the application with the VM argument -XX:+HeapDumpOnOutOfMemoryError.

Analyse the Heap Dump and find out what is causing the issue.

Eclipse MAT is an excellent tool for finding out such issues.

answered Aug 09 '12 at 13:48

Ajay George

11,113
1
37
46

Okay I'm going to go check out MAT now. – MalcolmOcean Aug 09 '12 at 14:10
I tried adding that argument to the VM, but if it's dumping the heap I can't tell where. – MalcolmOcean Aug 09 '12 at 17:23
It will be within your process dir and is configurable. http://stackoverflow.com/a/543037/628943 – Ajay George Aug 09 '12 at 17:30
Any luck with your analysis @MalcolmMcC ? – Ajay George Aug 12 '12 at 14:33
1

Ultimately it was a JRE problem. See my answer for more details. – MalcolmOcean Aug 15 '12 at 14:18

score 1 · Answer 2 · 2012-08-09T14:44:47.863

you need to setup the JVMs min and max heap memory

set JAVA_OPTS="-Xms128m -Xmx256m"

something like that but with bigger values like 2G, 4G whatever

LE: As you all know you can't force JVM to run the garbage collector (even though you can ask for it), but there are some ways of convincing it to get rid of some items by null-ing their references. Another thing to watch is the database object that might be lazy initialised. That error could appear when you try to create an object exceding the max heap memory.

Another ideea could be some retarded developer that programatically threw the OutOfMemoryError in some method for some retarded reason. When you reach that part of code, that's what you get (search the project)

It's not the heap size itself that's the problem... as I said, it happily expands much higher, but it shouldn't be expanding in the first place. Things are getting added to memory but not removed. — MalcolmOcean, Aug 09 '12 at 14:30

score 1 · Answer 3 · answered Aug 13 '12 at 14:27

There can be at least 2 reasons for the application to crash with OutOfMemoryError.

Your java heap is just too small for the amount of data it needs to process. Then you can either increase it as suggested Matei, or analyze heap dump as suggest Ajay.
Your application leaks memory. Which means that it leaves some unneeded data in memory after processing it. Then increasing heap will not help in the long run. And your options are either heap dump analysis (again) or specialised memory leak detection tool, such as Plumbr

score 1 · Accepted Answer · answered Aug 15 '12 at 14:16

Turned out the crash was caused by using the OpenJDK JRE rather than Oracle's JRE. I don't know exactly what the bug is in OpenJDK that makes it crash like this, but changing to Oracle's JRE ultimately solved the problem.

(I was using OpenJDK because I'm on a Linux computer that someone was using for open-source work before me. When I mentioned the crash to him he had the idea that that might be the cause. He was right.)

score 1 · Answer 5 · answered Oct 29 '13 at 10:41

Do you have a 32bit operative system without large memory support (PAE on win, huge mem kernel on linux...)? If yes, you may encounter the 2GB memory segment limit per process on 32 bit systems.

As a work around, try to set the JVM parameter -Xss192k to dispose 192kb of stack space per thread, and the parameter -Xmx1024m to use no more than 1GB of heap.

How am I getting a "java.lang.OutOfMemoryError: Java heap space" if my heap isn't full?

5 Answers5