Kallard
Kallard

Reputation: 31

SSL Handshake error after Client Finished handshake message

I'm attempting to perform mutual TLS authentication to server.com (changed), and I'm getting Fatal (HANDSHAKE_FAILURE): Couldn't kickstart handshaking, javax.net.ssl.SSLException: readHandshakeRecord after Produced client Finished handshake message.

I am using AdoptOpenJDK 11.0.11.9-hotspot.

JVM options:

-Djavax.net.ssl.trustStore=cacerts
-Djavax.net.ssl.trustStoreType=PKCS12
-Djavax.net.ssl.trustStorePassword=changeit
-Djavax.net.ssl.keyStore=keystore
-Djavax.net.ssl.keyStoreType=PKCS12
-Djavax.net.ssl.keyStorePassword=changeit
-Djavax.net.debug=SSL,handshake
-Djdk.tls.client.protocols=TLSv1.3
-Djdk.tls.server.protocols=TLSv1.3
-Dhttps.protocols=TLSv1.3

Code I use for debugging this problem:

public static void main (String[] args) throws IOException {
    URL url = new URL( "https://server.com");
    HttpURLConnection con = (HttpURLConnection) url.openConnection();
    con.setRequestMethod("GET");
    con.setConnectTimeout(5000);
    con.setReadTimeout(5000);
    con.connect();
}

Logs with handshake steps successfully performed:

javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.505 MSK|ClientHello.java:653|Produced ClientHello handshake message ("ClientHello": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.505 MSK|ServerHello.java:872|Consuming ServerHello handshake message ("ServerHello": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.505 MSK|ServerHello.java:968|Negotiated protocol version: TLSv1.3
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.536 MSK|ChangeCipherSpec.java:246|Consuming ChangeCipherSpec message
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.552 MSK|EncryptedExtensions.java:171|Consuming EncryptedExtensions handshake message (<...>)
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.552 MSK|CertificateRequest.java:926|Consuming CertificateRequest handshake message ("CertificateRequest": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.568 MSK|CertificateMessage.java:1171|Consuming server Certificate handshake message ("Certificate": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.677 MSK|CertificateVerify.java:1161|Consuming CertificateVerify handshake message ("CertificateVerify": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.677 MSK|Finished.java:898|Consuming server Finished handshake message ("Finished": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.693 MSK|CertificateMessage.java:1139|Produced client Certificate message ("Certificate": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.724 MSK|CertificateVerify.java:1126|Produced client CertificateVerify handshake message ("CertificateVerify": { <...> })
javax.net.ssl|DEBUG|01|main|2022-01-12 10:07:55.739 MSK|Finished.java:673|Produced client Finished handshake message ("Finished": { <...> })

Just after that I get:

javax.net.ssl|ERROR|01|main|2022-01-12 11:38:54.523 MSK|TransportContext.java:341|Fatal (HANDSHAKE_FAILURE): Couldn't kickstart handshaking (
"throwable" : {
  javax.net.ssl.SSLException: readHandshakeRecord
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1335)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)
    at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
    at SbbidIntegraionApplication.main(SbbidIntegraionApplication.java:17)
  Caused by: java.net.SocketException: Software caused connection abort: socket write error
    at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
    at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
    at java.base/sun.security.ssl.SSLSocketOutputRecord.flush(SSLSocketOutputRecord.java:251)
    at java.base/sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:89)
    at java.base/sun.security.ssl.Finished$T13FinishedProducer.onProduceFinished(Finished.java:679)
    at java.base/sun.security.ssl.Finished$T13FinishedProducer.produce(Finished.java:658)
    at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
    at java.base/sun.security.ssl.Finished$T13FinishedConsumer.onConsumeFinished(Finished.java:1011)
    at java.base/sun.security.ssl.Finished$T13FinishedConsumer.consume(Finished.java:874)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
    at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1418)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1324)
    ... 6 more}

)
javax.net.ssl|WARNING|01|main|2022-01-12 11:38:54.523 MSK|TransportContext.java:383|Fatal: failed to send fatal alert HANDSHAKE_FAILURE (
"throwable" : {
  java.net.SocketException: Software caused connection abort: socket write error
    at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
    at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
    at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:380)
    at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)
    at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
    at SbbidIntegraionApplication.main(SbbidIntegraionApplication.java:17)}

)
javax.net.ssl|DEBUG|01|main|2022-01-12 11:38:54.523 MSK|SSLSocketImpl.java:1638|close the underlying socket
javax.net.ssl|DEBUG|01|main|2022-01-12 11:38:54.523 MSK|SSLSocketImpl.java:1657|close the SSL connection (initiative)
Exception in thread "main" javax.net.ssl.SSLException: readHandshakeRecord
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1335)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440)
    at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411)
    at java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567)
    at java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:197)
    at java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:168)
    at SbbidIntegraionApplication.main(SbbidIntegraionApplication.java:17)
    Suppressed: java.net.SocketException: Software caused connection abort: socket write error
        at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
        at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
        at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
        at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:380)
        at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:292)
        at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:450)
        ... 5 more
Caused by: java.net.SocketException: Software caused connection abort: socket write error
    at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
    at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
    at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
    at java.base/sun.security.ssl.SSLSocketOutputRecord.flush(SSLSocketOutputRecord.java:251)
    at java.base/sun.security.ssl.HandshakeOutStream.flush(HandshakeOutStream.java:89)
    at java.base/sun.security.ssl.Finished$T13FinishedProducer.onProduceFinished(Finished.java:679)
    at java.base/sun.security.ssl.Finished$T13FinishedProducer.produce(Finished.java:658)
    at java.base/sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:436)
    at java.base/sun.security.ssl.Finished$T13FinishedConsumer.onConsumeFinished(Finished.java:1011)
    at java.base/sun.security.ssl.Finished$T13FinishedConsumer.consume(Finished.java:874)
    at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:443)
    at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:421)
    at java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:182)
    at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171)
    at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1418)
    at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1324)
    ... 6 more

Process finished with exit code 1

The same request sent with curl or openssl is processed successfully and I get an expected answer from the server.

What can be the reason for such an error almost at the end of the handshake process?

Thanks


UPD:

After comment of @dkotenko I initialised SSLContext programmatically:

public static void main (String []args) throws IOException {

    KeyStore keyStore = KeyStore.getInstance("PKCS12");
    keyStore.load(new FileInputStream("keystore"), "changeit".toCharArray());
    KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
    keyManagerFactory.init(keyStore, "changeit".toCharArray());
    
    KeyStore trustStore = KeyStore.getInstance("PKCS12");
    trustStore.load(new FileInputStream("cacerts"), "changeit".toCharArray());
    TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
    trustManagerFactory.init(trustStore);
    
    SSLContext sslContext = SSLContext.getInstance("TLS");
    sslContext.init(keyManagerFactory.getKeyManagers(), trustManagerFactory.getTrustManagers(), new SecureRandom());
    
    HttpsURLConnection.setDefaultSSLSocketFactory(sslContext.getSocketFactory());
    URL url = new URL("server.com");
    HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
    connection.setRequestMethod("GET");
    connection.connect();
}

but got exactly the same output.

Upvotes: 1

Views: 13079

Answers (1)

Kallard
Kallard

Reputation: 31

Solved. The problem was in the absence of CA certificates in the keystore.

The server administrator said that when contacting the load balancer at server.com there is a check that there is a certificate of the particular root CA in the client's certificate chain.

After importing it into the keystore, the entire handshake process was complete and a correct response was received from the server

Hope this will help somebody one day:)

Upvotes: 2

Related Questions