TrustStore fails to init but KeyStore successful

Question

I am trying to configure Tomcat so that my web application can use LDAPS. It all works fine on the UAT server but for some reason the production server cannot initialize the trust store. I have turned on debugging (via -Djavax.net.debug=all) and the error is:

default context init failed: java.io.IOException: Keystore was tampered with, or password was incorrect

I am 100% sure that the password is correct, as I have run keytool and listed the contents of the truststore, as well as taken the same file from the working UAT server.

As an experiment, I configured Tomcat to use the same store file for both keystore and truststore (I realise this is not a typical setup, but from my understanding the file formats etc are the same so they should both load):

-Djavax.net.ssl.trustStorePassword=xxx -Djavax.net.ssl.trustStore=C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin\.keystore -Djavax.net.ssl.keyStorePassword=xxx -Djavax.net.ssl.keyStore=C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin\.keystore

and the keystore works:

keyStore is : C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin.keystore
keyStore type is : jks
keyStore provider is :
init keystore
init keymanager of type SunX509 ***
found key for : tomcat

whilst the truststore fails.. for the same file and password!:

trustStore is: C:\Program Files\Apache Software Foundation\Tomcat 7.0\bin.keystore
trustStore type is : jks
trustStore provider is :
init truststore
default context init failed: java.io.IOException: Keystore was tampered with, or password was incorrect

From the web application the stack trace has some more detail (whether it helps I'm not sure.. I studied the java source for TrustManagerFactoryImpl and matched it with the logging but it didn't explain the behaviour)

javax.naming.CommunicationException: <ldap url>:636 [Root exception is java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)]
    at com.sun.jndi.ldap.Connection.<init>(Unknown Source)
    at com.sun.jndi.ldap.LdapClient.<init>(Unknown Source)
    at com.sun.jndi.ldap.LdapClient.getInstance(Unknown Source)
    at com.sun.jndi.ldap.LdapCtx.connect(Unknown Source)
    at com.sun.jndi.ldap.LdapCtx.<init>(Unknown Source)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURL(Unknown Source)
    at com.sun.jndi.ldap.LdapCtxFactory.getUsingURLs(Unknown Source)
    at com.sun.jndi.ldap.LdapCtxFactory.getLdapCtxInstance(Unknown Source)
    at com.sun.jndi.ldap.LdapCtxFactory.getInitialContext(Unknown Source)
    at javax.naming.spi.NamingManager.getInitialContext(Unknown Source)
    at javax.naming.InitialContext.getDefaultInitCtx(Unknown Source)
    at javax.naming.InitialContext.init(Unknown Source)
    at javax.naming.InitialContext.<init>(Unknown Source)
    at javax.naming.directory.InitialDirContext.<init>(Unknown Source)
    at com.mycompany.fde.util.LDAPHelper.initialBindSSL(LDAPHelper.java:116)
    at com.mycompany.fde.util.LDAPHelper.<init>(LDAPHelper.java:91)
    at com.mycompany.fde.util.LDAPHelper.getInstance(LDAPHelper.java:58)
    at com.mycompany.fde.server.work.StartupWork.startupStage1(StartupWork.java:224)
    at com.mycompany.fde.server.work.StartupWork.call(StartupWork.java:121)
    at com.mycompany.fde.server.work.StartupWork.call(StartupWork.java:47)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.net.SocketException: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
    at javax.net.ssl.DefaultSSLSocketFactory.throwException(Unknown Source)
    at javax.net.ssl.DefaultSSLSocketFactory.createSocket(Unknown Source)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at com.sun.jndi.ldap.Connection.createSocket(Unknown Source)
    ... 24 more
Caused by: java.security.NoSuchAlgorithmException: Error constructing implementation (algorithm: Default, provider: SunJSSE, class: sun.security.ssl.SSLContextImpl$DefaultSSLContext)
    at java.security.Provider$Service.newInstance(Unknown Source)
    at sun.security.jca.GetInstance.getInstance(Unknown Source)
    at sun.security.jca.GetInstance.getInstance(Unknown Source)
    at javax.net.ssl.SSLContext.getInstance(Unknown Source)
    at javax.net.ssl.SSLContext.getDefault(Unknown Source)
    at javax.net.ssl.SSLSocketFactory.getDefault(Unknown Source)
    ... 29 more
Caused by: java.io.IOException: Keystore was tampered with, or password was incorrect
    at sun.security.provider.JavaKeyStore.engineLoad(Unknown Source)
    at sun.security.provider.JavaKeyStore$JKS.engineLoad(Unknown Source)
    at java.security.KeyStore.load(Unknown Source)
    at sun.security.ssl.TrustManagerFactoryImpl.getCacertsKeyStore(Unknown Source)
    at sun.security.ssl.SSLContextImpl$DefaultSSLContext.getDefaultTrustManager(Unknown Source)
    at sun.security.ssl.SSLContextImpl$DefaultSSLContext.<init>(Unknown Source)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
    at java.lang.reflect.Constructor.newInstance(Unknown Source)
    ... 35 more
Caused by: java.security.UnrecoverableKeyException: Password verification failed
    ... 45 more

I'm at a complete loss - any help/ideas would be much appreciated!

Answer 1

For the sake of completeness, this can also occur if the identically configured keystore file does not exist (here: erreneous file:/// prefix). It still leads to:

trustStore is: No File Available, using empty keystore.
trustStore type is : jks
trustStore provider is : 
init truststore
keyStore is : file:///c:/my.jks
keyStore type is : jks
keyStore provider is : 

Compared to OpenJDK 1.8.0.181, the keyStore is : line just misleadingly outputs javax.net.ssl.keyStore [1] while the given trustStore is checked for existance first [2].

[1] sun.security.ssl.SSLContextImpl
[2] sun.security.ssl.TrustManagerFactoryImpl

Answer 2

Thanks everyone for your help, I finally found that for some unknown reason, someone had indeed set the truststore password in Catalina.properties (via javax.net.ssl.trustStorePassword). It seems that this takes priority even over the java variables set in the Tomcat UI :(

Answer 3

You don't need to supply the truststore password. So, don't.

NB keytool -list doesn't need the password at all, and it isn't proof you know the correct password. The error you're getting, on the other hand, is proof that the password is incorrect.

You should not use the same file for both keystore and truststore.