4

I am running an Apache Storm topology on a server using a Java 7 JVM. I've been considering some JVM tuning and noticed it is currently using a Concurrent Mark and Sweep (CMS) garbage collector. This makes sense, as the server has 32 cores and while it is running multiple JVMs with this setting, it is only running 4 such JVMs which is less than 32 cores.

However, I noticed we're running the garbage collector with the setting CMSConcurrentMTEnabled turned off. The default is for that setting to be turned on, which makes me wonder why anyone would choose to use a single thread for concurrent garbage collection when other threads are available. Under what conditions does using that setting make sense, supposing other threads are available?

(Editing for more detail on my current situation, with hopes that it will lead to an answer) :

The JVMs seem to be running out of memory, performing minor GCs repeatedly that are taking ~9 seconds each, and eventually crashing. No OutOfMemoryError has been thrown, which baffles me. The minor GCs are cleaning up a few kB each time and the overall usage hovers around 100% for quite a while. The heap size is 4 GB so those conditions should trigger an OutOfMemoryError. However, this is getting into a very localized situation for an SO question, I fear.

Aditya W
  • 636
  • 8
  • 18
BlackVegetable
  • 10,734
  • 6
  • 45
  • 73
  • Obvious question: do you have any performance or memory problems which have prompted you in the past to change the default settings of the JVM? – fge Feb 13 '16 at 20:09
  • I'm looking into that; I wasn't the one that set up the original settings. We currently are having memory issues that I'm investigating. – BlackVegetable Feb 13 '16 at 22:04
  • Can you define "memory issues"? Are those of the kind which trigger an `OutOfMemoryError` on a regular basis? Do you have a test environment on which you can stress test your environment? – fge Feb 13 '16 at 22:08

2 Answers2

2

So, according to experts on Twitter and some mailing lists, there is a bug in Java 6 update 21 where

...CMS won't always free objects with finalizers[.]

The fix is to disable the setting CMSConcurrentMTEnabled. (Or use a newer version of Java.) I am not certain which version of Java was the first to fix this issue.

BlackVegetable
  • 10,734
  • 6
  • 45
  • 73
0

EDIT:

Using a single thread on multi core machine does not use full capabilities of machine.

If you want to increase number of parallel gc threads with CMS, you can use below configuration.

The number of garbage collector threads can be controlled with the command line option -XX:ParallelGCThreads=.

More fine tuning options can be found at documentation article.

Consider using G1GC in place of CMS.

G1GC has become very popular with large heaps post JDK 7 update 4 release. It may become default GC algorithm in Java 9 version.

By setting goals like pause time goal, you can improve predictability of pause time.

Have a look at

http://www.oracle.com/technetwork/articles/java/g1gc-1984535.html

and

http://www.oracle.com/technetwork/tutorials/tutorials-1876574.html

Aditya W
  • 636
  • 8
  • 18