4

I have a JEE application that has recently started to see spikes in CPU usage (e.g. 100% of 27 cores on a 40 core server) and increasingly longer periods of time where the application is unavailable. It's very similar behavior to the issue described in the following post, to include the fact that bouncing the application server makes the issue go away until it appears again after a few hours:

Old Gen heap is full and the Eden and Survivor are low and almost empty

I've taken some core dump outputs while the application is experiencing these "freezes" and I am seeing the following JVM GC output:

PSYoungGen total 11221504K, used 2435K
eden space 9238528K, 0% used
from space 19829796K, 0% used
to space 1970176K, 0% used
ParOldGen total 39613440K, used 39276477K
object space 39613440K, 99% used
PSPermGen total 254976K, used 115497K
object space 254976K, 45% used

Based on the referenced post and the above output, I think I understand that the "freezes" are being driven by the garbage collector running (in vain?) on the ParOldGen space. The parts I am missing:

  1. Why the PermGen space remains at 45% used. That is, will the ~39GB of stuff in ParOldGen ultimately transition into PSPermGen?

  2. What is the significance of the nearly empty PSYoungGen space? Does this mean that the application isn't creating any/many new object instances at steady state?

The post above also describes the option of "giving more headroom" to ParOldGen, but I'm not clear if that means increasing the total heap size via -Xmx or if there's an explicit JVM GC parameter. I see the NewRatio argument controls the size of the young generation relative to the old generation. Would the fact that the PSYoungGen is essentially empty mean that it's too large, and that I should use a smaller NewRatio value?

Thanks in advance for any assistance.

Community
  • 1
  • 1
Andrew David
  • 81
  • 1
  • 5
  • stuff from old gen doesn't get promoted to permgn space, you probably have a leak somewhere in permgen, does your app create lots of classes dynamically ? – jmj May 15 '16 at 05:38
  • Thank you for your comment. I now understand that permgen space is not part of the heap (per http://stackoverflow.com/questions/2129044/java-heap-terminology-young-old-and-permanent-generations ). The app does create a lot classes dynamically. It seems we either have a memory leak or the app's memory requirements are exceeding the current -Xmx value. – Andrew David May 15 '16 at 14:18
  • You create a lot of classes or objects dynamically? If classes then your permgen will get full, if objects then oldgen. – Krzysztof Krasoń May 15 '16 at 16:23
  • those class definition goes into PermGen space, if you create lots of them you are likely to hit the MaxPermSize and so the slowness and OOM eventually, looking at the data again, your permgen isn't looking into trouble zone, it is your old gen – jmj May 15 '16 at 21:46
  • Almost 40Gb of heap space for OldGen + about 10Gb for the YoungGen, means you are running with a 50Gb heap. This sounds a lot to me. Can you provide some details why do you need so much? Anyway, I think first of all you need to generate a heap dump and analyze it with MAT or anything else. Get an understanding what objects are eating up your heap, do you really need them in the heap, and if not see why those are not GC-ed. – Vladimir G. May 16 '16 at 18:06

2 Answers2

1

will the ~39GB of stuff in ParOldGen ultimately transition into PSPermGen?

The PermGen in Java 7 (replaced with metaspace in Java 8) is for holding code. The only things which pass from the heap to PermGen is byte code, so unless you are generating or loading classes, nothing passes from one to the other. They are different spaces.

What is the significance of the nearly empty PSYoungGen space?

The young gen is empty after a full GC. Full GCs are common once your old gen starts to fill up.

Does this mean that the application isn't creating any/many new object instances at steady state?

It is more likely to mean it has recently Full GC-ed.

describes the option of "giving more headroom" to ParOldGen, but I'm not clear if that means increasing the total heap size via -Xmx or if there's an explicit JVM GC parameter.

Increasing the maximum heap could give you more head room, but I would first check

  • you don't have a memory leak.
  • you can't move the bulk of your data off heap e.g. a database or native memory.

Would the fact that the PSYoungGen is essentially empty mean that it's too large, and that I should use a smaller NewRatio value?

That could help give you more space for the old gen, but it might just give you a bit more time before it runs out of memory.

Peter Lawrey
  • 498,481
  • 72
  • 700
  • 1,075
0

Why the PermGen space remains at 45% used. That is, will the ~39GB of stuff in ParOldGen ultimately transition into PSPermGen?

No. OldGen & PermGen spaces can't be shared. PSPermGen mostly contains classes loaded by class loader in PermGen space. ParOldGen contains heap memory for long lived objects.

In JDK 1.8, PermGen has replaced by Metaspace. Refer to this article for more details.

Refer to below SE questions for more details:

Java heap terminology: young, old and permanent generations?

What is the significance of the nearly empty PSYoungGen space? Does this mean that the application isn't creating any/many new object instances at steady state?

@Peter Lawrey has correctly answered it. After Full GC, YoungGen is nearly empty => you did not accumulate garbage from short lived objects in your application

The post above also describes the option of "giving more headroom" to ParOldGen, but I'm not clear if that means increasing the total heap size via -Xmx or if there's an explicit JVM GC parameter.

Your old gen is full implies that your application is retaining heap from long lived objects.

Now you have to check possible memory leaks (if any) in your application using profiling tools like visualvm or MAT

If you don't have memory leaks, you can increase your heap size with -Xmx.

Would the fact that the PSYoungGen is essentially empty mean that it's too large, and that I should use a smaller NewRatio value?

Since you are using larger heaps, I recommend you to use G1GC algorithm. If you use G1GC algorithm, don't customize default values since G1GC algorithm takes care of better heap management.

Have a look at oracle article on G1GC switches (Complete List of G1 GC Switches section), Use Cases article and related SE question:

Java 7 (JDK 7) garbage collection and documentation on G1

Community
  • 1
  • 1
Ravindra babu
  • 42,401
  • 8
  • 208
  • 194