Oracle JDBC pool SocketException: Connection reset on startup

Question

I used to have several connection reset errors on startup of my application.
What are possible cause and how could I fix this?

    ...
Caused by: java.net.SocketException: Connection reset
    at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:115) ~[?:1.8.0_121]
    at java.net.SocketOutputStream.write(SocketOutputStream.java:155) ~[?:1.8.0_121]
    at oracle.net.ns.DataPacket.send(DataPacket.java:209) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ano.CryptoDataPacket.send(Unknown Source) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ns.NetOutputStream.flush(NetOutputStream.java:215) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:302) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:249) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:171) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.net.ns.NetInputStream.read(NetInputStream.java:89) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:123) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:79) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:429) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:397) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:437) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:954) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:639) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:666) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:566) ~[ojdbc7-12.1.0.2.jar:12.1.0.2.0]
    at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:310) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:203) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:732) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:664) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:479) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:154) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:118) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:107) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:131) ~[tomcat-jdbc-8.5.11.jar:?]
    at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
    at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:326) ~[spring-jdbc-4.3.6.RELEASE.jar:4.3.6.RELEASE]
    ... 61 more

Answer 1

Adding JVM system propery may help:

-Djava.security.egd=file:/dev/../dev/urandom

java.security.SecureRandom is a standard API provided by sun. Among various methods offered by this class void nextBytes(byte[]) is one. This method is used for generating random bytes. Oracle 11g JDBC drivers use this API to generate random number during login. Users using Linux have been encountering SQLException("Io exception: Connection reset").

The problem is two fold

1. The JVM tries to list all the files in the /tmp (or alternate tmp directory set by -Djava.io.tmpdir) when SecureRandom.nextBytes(byte[]) is invoked. If the number of files is large the method takes a long time to respond and hence cause the server to timeout

2. The method void nextBytes(byte[]) uses /dev/random on Linux and on some machines which lack the random number generating hardware the operation slows down to the extent of bringing the whole login process to a halt. Ultimately the the user encounters SQLException("Io exception: Connection reset")

Users upgrading to 11g can encounter this issue if the underlying OS is Linux which is running on a faulty hardware.

Cause The cause of this has not yet been determined exactly. It could either be a problem in your hardware or the fact that for some reason the software cannot read from dev/random

(source)