Start Java/Play2.5 REST API without DB

Question

I would like to start my Java/Play! Service even though the Database is not yet reachable. So far, if the database is not available the service stops. Is it possible in Play to try and reconnect every second or so without completely stopping the service and reconnecting to db as soon as connection is available.

Our application.conf currently looks like this:

play.evolutions.enabled=false
//play.db.pool=bonecp

db {
   default.driver=org.postgresql.Driver
   default.url="jdbc:postgresql://postgresdb.postgres:5432/postcode"
   default.username="<username>"
   default.password="<strong-password>"
}

play.db {
  config = "db"
  prototype.hikaricp = {
    connectionTimeout = 250 milliseconds
    initializationFailFast = false
    readOnly = true
  }
}

ebean.default = ["models.*"]

Currently I get the following logs:

2017-05-15 13:26:54.525 [error] postcode-play: Connection error: 
org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
  Ort: Datei: postinit.c, Routine: InitPostgres, Zeile: 779.
  Server SQLState: 53300
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412)
    at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2538)
    at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:122)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:227)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:431)
    at org.postgresql.Driver.connect(Driver.java:247)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)

2017-05-15 13:26:54.626 [error] postcode-play: Connection error: 
org.postgresql.util.PSQLException: FATAL: remaining connection slots are reserved for non-replication superuser connections
  Ort: Datei: postinit.c, Routine: InitPostgres, Zeile: 779.
  Server SQLState: 53300
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2412)
    at org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2538)
    at org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:122)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:227)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:431)
    at org.postgresql.Driver.connect(Driver.java:247)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
.
.
.

Indicating that the server does not stop altogether but tries to connect to db multiple times which is as desired. But if the database becomes available the following logs are written:

2017-05-15 14:08:43.748 [error] postcode-play: 

! @7430ed7gf - Internal server error, for (GET) [/?postcode=88682&country=DE] ->

play.api.Configuration$$anon$1: Configuration error[Cannot connect to database [default]]
    at play.api.Configuration$.configError(Configuration.scala:155)
    at play.api.Configuration.reportError(Configuration.scala:808)
    at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:48)
    at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:42)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at play.api.db.DefaultDBApi.connect(DefaultDBApi.scala:42)
    at play.api.db.DBApiProvider.get$lzycompute(DBModule.scala:72)
    at play.api.db.DBApiProvider.get(DBModule.scala:62)
    at play.api.db.DBApiProvider.get(DBModule.scala:58)
    at com.google.inject.internal.ProviderInternalFactory.provision(ProviderInternalFactory.java:81)
Caused by: java.sql.SQLTransientConnectionException: HikariPool-4 - Connection is not available, request timed out after 5001ms.
    at com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:548)
    at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:186)
    at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:145)
    at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:83)
    at play.api.db.DefaultDatabase.getConnection(Databases.scala:142)
    at play.api.db.DefaultDatabase.getConnection(Databases.scala:138)
    at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:44)
    at play.api.db.DefaultDBApi$$anonfun$connect$1.apply(DefaultDBApi.scala:42)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at play.api.db.DefaultDBApi.connect(DefaultDBApi.scala:42)
Caused by: org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:265)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194)
    at org.postgresql.Driver.makeConnection(Driver.java:431)
    at org.postgresql.Driver.connect(Driver.java:247)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:95)
    at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:101)
    at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:341)
    at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:193)
    at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:430)
Caused by: java.net.ConnectException: Verbindungsaufbau abgelehnt (Connection refused)
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:589)
    at org.postgresql.core.PGStream.<init>(PGStream.java:62)
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:144)
    at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:49)
    at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:194
)

The server cannot reach out to the database even though it is available now. If I restart the server the database will be found and can be worked with, but the database lways has to be there on startup. How can I fix that, so that the app can be started independently of the database?

If the database becomes unavailable after the application has started it is no problem after the db becomes available again. it is automatically reconnected.

Any help is greatly apreciated!

score 2 · Accepted Answer · answered May 22 '17 at 18:32

It seems like you are leaving some connections open to Postgres. These connections must be closed or else Postgres will hit its max_connections limit.

You could increase max_connections but it's not recommended as performance of Postgres will degrade if it's set too high. Also, as you are using HikariCP it wouldn't be a bad idea to take a look at the HikariCP docs, in particular the idleTimeout and the minimumIdle config variables should help you.

If the above doesn't work then you must confirm exactly how many connections are being made to Postgres. Maybe even check the number of sessions you personally are making to Postgres via SSH or through other apps, not just the number of Play Framework connections.

Start Java/Play2.5 REST API without DB

1 Answers1