8

I have a web application which often takes an inordinate amount of time to deploy on Tomcat. My suspicion is that there's a database connection somewhere that's waiting until a timeout, but that's just a guess and I want to find out for sure what's causing the hold up so I can remedy the issue. Can anyone suggest a way I can go about doing this? Should I profile Tomcat when it's loading the WAR and look there for clues? If so is there a tutorial somewhere that's good for a beginner?

In case this matters my web application uses Spring and Hibernate. I've been told by a colleague that perhaps these are causing the slow down in that they are so huge that somewhere a class loader is choking on the sheer number of classes it needs to load.

Also I see this when I stop Tomcat or hot deploy the WAR to an already running Tomcat:

Jun 1, 2012 6:03:33 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/nacem-rest] registered the JDBC driver [oracle.jdbc.OracleDriver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jun 1, 2012 6:03:34 PM org.apache.catalina.startup.HostConfig deployWAR

Perhaps this is part of the problem? Redeploying my web application almost always requires a restart of Tomcat as well, and I've always assumed that the above mentioned JDBC driver issue was to blame, but maybe it also has something to do with the sluggish start time as well?

James Adams
  • 7,338
  • 18
  • 68
  • 113

7 Answers7

8

You can use a profiler, but it is a bit of an overkill. Instead just make few thread dumps while the application loads. You can use jstack or jvisualvm, plenty of resources over the net describing how to do this. Basically you need a PID of your Tomcat (see: jps).

If you find it hard to analyse the thread dump - add it to your question. Typically it's pretty easy to find a bottleneck. Typical problems:

  • the application tries to fetch some resource from the Internet like XML schema
  • a lot of classes to scan on the CLASSPATH
  • excessive GC (enable verbose gc logging just in case)
  • not enough memory (swapping, see iostat/iotop)
Tomasz Nurkiewicz
  • 311,858
  • 65
  • 665
  • 652
5

the first I would look is how the CPU or RAM usage of tomcat perform during your application startup.

If you see a lot of CPU activity and RAM increase, then it's probably loading a lot of stuff, like a lot of classes, or performing some kind of huge pre-allocation or similar. In that case thread dumps can help you a lot, but also "lsof" (if your tomcat is running on an *nix environment) to see what files it is currently working on.

However, if you see it simply sitting there, then it is probably waiting on some kind of connection.

One cause may be a DB connection, as you supposed. The DB is usually fast to answer, and a failed attempt to connect to a DB is usually clearly logged somewhere, but still it could be.

Another, less known, cause may be some XML validation. It is not uncommon for a web application to load some XML data, and it is not uncommon to use a validating parser to load that XML. The validating parser needs a schema or DTD to validate, and the URL for the schema/DTD file is often included in the XML. So, some parsers will try to load the schema file from internet to validate the XML, and that could take a lot of time being an internet connection. Moreover, some parsers will fail silently and simply not validate the XML, probably after a rather long timeout. Sorry for being vague on "some parsers" etc.. but if XML loading is done by libraries used inside your webapp, they could use the JVM XML parser, any possible version of Xerces, any possible version for JDOM etc... and even worst different libraries may be using different parsers.

However, if it is a connection problem, it's easier to see it using "netstat -anlp | grep java" (in your tomcat is on a *nix environment, otherwise it should be "netstat -ano" on windows), and looking for what outgoing connection your tomcat is trying to make. There you can see DB connections and also outgoing (usually http) connections searching for schemes or other stuff.

Simone Gianni
  • 10,588
  • 36
  • 45
4

If you are using Tomcat 7.0.x, it might be scanning the classpath to detect annotations. If you have a lot of jars / classes, then this can be an issue.

As Simone says, taking and analysing a couple of thread dumps is a good way to start.

If you are not using a recent version of Tomcat, try upgrading & checking the conf/catalina.properties file for hints on how to reduce the number of jars that are scanned.

Pidster
  • 628
  • 4
  • 9
3

You should take a few thread dumps during deployment (the more the better), and analyze them. Look for blocks, waits etc. Thread Dump Analyzer might be of use.

npe
  • 14,537
  • 1
  • 51
  • 54
1

This may be relate to this BUG of JDK

I readed this article, Tomcat7 starts too late on Ubuntu 14.04 x64 and resolved my problem.

Try to resolve it by Replacing securerandom.source=file:/dev/urandom with securerandom.source=file:/dev/./urandom in $JAVA_PATH/jre/lib/security/java.security

Community
  • 1
  • 1
Tuấn Lé
  • 11
  • 3
  • This helped to eradicate a 7-minute delay in deployment because of delays relating to `createSecureRandom` – ATG Sep 22 '15 at 13:45
0

I found that I had a large delay when deploying a web application to Tomcat8. I had three database connection pools configured in my context.xml file within the WAR. Each connection pool had an initial count of 10 connections so Tomcat was creating 30 database connections at startup. I found that configuring each pool for 1 initial connection on my test system improved the deploy time from approximately 3 minutes to 22 seconds.

user3590949
  • 1
-3

Append below in setenv.sh of your working tomcat7/bin directory:

JAVA_OPTS="-Djava.security.egd=file:/dev/./urandom"

Enwired
  • 1,503
  • 12
  • 24
Sangameshrh
  • 21
  • 1