HTTPS connection with client certificate in an android app

Question

I am trying to replace the currently working HTTP connection with a HTTPS connection in a Android app that I am writing. The additional security of a HTTPS connection is necessary and so I cannot ignore this step.

I have the following:

1. A server configured to establish a HTTPS connection, and require a client certificate
   • This server has a certificate that is issued by a standard large-scale CA. In short, if I access this connection via the browser in Android, it works fine because the devices truststore recognizes the CA. (So it's not self-signed)
2. A client certificate that is essentially self-signed. (Issued by an internal CA)
3. An Android app that loads this client certificate and attempts to connect to the aforementioned server, but has the following problems/properties:
   • The client can connect to the server when the server is configured to not require a client certificate. Basically, if I use SSLSocketFactory.getSocketFactory() the connection works fine, but the client certificate is a required part of this applications specifications, so:
   • The client produces a javax.net.ssl.SSLPeerUnverifiedException: No peer certificate exception when I attempt to connect with my custom SSLSocketFactory, but I am not entirely certain why. This exception seems a little ambiguous after searching around the internet for various solutions to this.

Here is the relavent code for the client:

SSLSocketFactory socketFactory = null;

public void onCreate(Bundle savedInstanceState) {
    loadCertificateData();
}

private void loadCertificateData() {
    try {
        File[] pfxFiles = Environment.getExternalStorageDirectory().listFiles(new FileFilter() {
            public boolean accept(File file) {
                if (file.getName().toLowerCase().endsWith("pfx")) {
                    return true;
                }
                return false;
            }
        });

        InputStream certificateStream = null;
        if (pfxFiles.length==1) {
            certificateStream = new FileInputStream(pfxFiles[0]);
        }

        KeyStore keyStore = KeyStore.getInstance("PKCS12");
        char[] password = "somePassword".toCharArray();
        keyStore.load(certificateStream, password);

        System.out.println("I have loaded [" + keyStore.size() + "] certificates");

        KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
        keyManagerFactory.init(keyStore, password);

        socketFactory = new SSLSocketFactory(keyStore);
    } catch (Exceptions e) {
        // Actually a bunch of catch blocks here, but shortened!
    }
}

private void someMethodInvokedToEstablishAHttpsConnection() {
    try {
        HttpParams standardParams = new BasicHttpParams();
        HttpConnectionParams.setConnectionTimeout(standardParams, 5000);
        HttpConnectionParams.setSoTimeout(standardParams, 30000);

        SchemeRegistry schRegistry = new SchemeRegistry();
        schRegistry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
        schRegistry.register(new Scheme("https", socketFactory, 443));
        ClientConnectionManager connectionManager = new ThreadSafeClientConnManager(standardParams, schRegistry);

        HttpClient client = new DefaultHttpClient(connectionManager, standardParams);
        HttpPost request = new HttpPost();
        request.setURI(new URI("https://TheUrlOfTheServerIWantToConnectTo));
        request.setEntity("Some set of data used by the server serialized into string format");
        HttpResponse response = client.execute(request);
        resultData = EntityUtils.toString(response.getEntity());
    } catch (Exception e) {
        // Catch some exceptions (Actually multiple catch blocks, shortened)
    }
}

I have verified that, yes indeed the keyStore loads a certificate and is all happy with that.

I have two theories as to what I'm missing from reading about HTTPS/SSL connections, but as this is really my first foray, I am a little puzzled as to what I actually need to resolve this issue.

The first possibility, as far as I can tell, is that I need to configure this SSLSocketFactory with the devices' truststore that includes all of the standard Intermediate and endpoint Certificate Authorities. That is, the device's default of SSLSocketFactory.getSocketFactory() loads some set of CAs into the factory's truststore that is used to trust the server when it sends its certificate, and that is what is failing in my code, because I do not properly have the trust store loaded. If this is true, how would I best go about loading this data?

The second possibility is due to the fact that the client certificate is self-signed (or issued by an internal certificate authority -- correct me if I'm wrong, but these really amount to the same thing, for all intents and purposes here). It is in fact this truststore that I am missing, and basically I need to provide a way for the server to validate the certificate with the internal CA, and also validate that this internal CA is in fact "trustable". If this is true, exactly what sort of thing am I looking for? I have seen some reference to this that makes me believe this may be my problem, as in here, but I am truly not certain. If this is indeed my problem, what would I ask for from the person who maintains the internal CA, and then how would I add this to my code so that my HTTPS connection would work?

The third, and hopefully less possible solution, is that I'm entirely wrong about some point here and have missed a crucial step or am completely neglecting a portion of HTTPS/SSL that I just don't currently have any knowledge of. If this is the case, could you please provide me with a bit of a direction so that I can go and learn what it is I need to learn?

Thanks for reading!

Answer 1

I had tried a couple of days I'm finally get the answer so I would like to post here my steps and all my code in order to help someone else.

1) to get the certificate of the site you want to connect

echo | openssl s_client -connect ${MY_SERVER}:443 2>&1 |  sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > mycert.pem

2)to create your key you need BouncyCastle library you can download here

keytool -import -v -trustcacerts -alias 0 -file mycert.pem -keystore “store_directory/mykst“ -storetype BKS -provider org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath “directory_of_bouncycastle/bcprov-jdk16-145.jar” -storepass mypassword

3) to check if the key was created

keytool -list -keystore "carpeta_almacen/mykst" -provider org.bouncycastle.jce.provider.BouncyCastleProvider -providerpath "directory_of_bouncycastle/bcprov-jdk16-145.jar" -storetype BKS -storepass mypassword

and you should see something like this :

Tipo de almacén de claves: BKS Proveedor de almacén de claves: BC

Su almacén de claves contiene entrada 1

0, 07-dic-2011, trustedCertEntry,

Huella digital de certificado (MD5):

55:FD:E5:E3:8A:4C:D6:B8:69:EB:6A:49:05:5F:18:48

4)then you need to copy the file "mykst" into the directory "res/raw" (create it if does not exist) in your android project.

5)add the permissions to in the android manifest

  <uses-permission android:name="android.permission.INTERNET"/>

6) here the code!

activity_main.xml

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:orientation="vertical"
    android:padding="10dp" >

    <Button
        android:id="@+id/button"
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:text="Cargar contenido" />

    <RelativeLayout
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:background="#4888ef">
        <ProgressBar
            android:id="@+id/loading"
            android:layout_width="50dp"
            android:layout_height="50dp"
            android:indeterminate="true"
            android:layout_centerInParent="true"
            android:visibility="gone"/>
        <ScrollView
            android:layout_width="fill_parent"
            android:layout_height="fill_parent"
            android:fillViewport="true"
            android:padding="10dp">
            <TextView
                android:id="@+id/output"
                android:layout_width="fill_parent"
                android:layout_height="fill_parent"
                android:textColor="#FFFFFF"/>
        </ScrollView>
    </RelativeLayout>
</LinearLayout>

MyHttpClient

package com.example.https;


import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.cert.X509Certificate;
import java.util.Date;
import java.util.Enumeration;

import org.apache.http.conn.ClientConnectionManager;
import org.apache.http.conn.scheme.PlainSocketFactory;
import org.apache.http.conn.scheme.Scheme;
import org.apache.http.conn.scheme.SchemeRegistry;
import org.apache.http.conn.ssl.SSLSocketFactory;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.impl.conn.SingleClientConnManager;

import android.content.Context;
import android.os.Build;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.TrustManagerFactory;

public class MyHttpClient extends DefaultHttpClient {

    final Context context;

    public MyHttpClient(Context context) {
        this.context = context;
    }

    @Override
    protected ClientConnectionManager createClientConnectionManager() {
        SchemeRegistry registry = new SchemeRegistry();
        registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80));
        // Register for port 443 our SSLSocketFactory with our keystore
        // to the ConnectionManager
        registry.register(new Scheme("https", newSslSocketFactory(), 443));
        return new SingleClientConnManager(getParams(), registry);
    }

    private SSLSocketFactory newSslSocketFactory() {
        try {
            // Trust manager / truststore
            KeyStore trustStore=KeyStore.getInstance(KeyStore.getDefaultType());

            // If we're on an OS version prior to Ice Cream Sandwich (4.0) then use the standard way to get the system
            //   trustStore -- System.getProperty() else we need to use the special name to get the trustStore KeyStore
            //   instance as they changed their trustStore implementation.
            if (Build.VERSION.RELEASE.compareTo("4.0") < 0) {
                TrustManagerFactory trustManagerFactory=TrustManagerFactory
                        .getInstance(TrustManagerFactory.getDefaultAlgorithm());
                FileInputStream trustStoreStream=new FileInputStream(System.getProperty("javax.net.ssl.trustStore"));
                trustStore.load(trustStoreStream, null);
                trustManagerFactory.init(trustStore);
                trustStoreStream.close();
            } else {
                trustStore=KeyStore.getInstance("AndroidCAStore");
            }

            InputStream certificateStream = context.getResources().openRawResource(R.raw.mykst);
            KeyStore keyStore=KeyStore.getInstance("BKS");
            try {
                keyStore.load(certificateStream, "mypassword".toCharArray());
                Enumeration<String> aliases=keyStore.aliases();
                while (aliases.hasMoreElements()) {
                    String alias=aliases.nextElement();
                    if (keyStore.getCertificate(alias).getType().equals("X.509")) {
                        X509Certificate cert=(X509Certificate)keyStore.getCertificate(alias);
                        if (new Date().after(cert.getNotAfter())) {
                            // This certificate has expired
                            return null;
                        }
                    }
                }
            } catch (IOException ioe) {
                // This occurs when there is an incorrect password for the certificate
                return null;
            } finally {
                certificateStream.close();
            }

            KeyManagerFactory keyManagerFactory=KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
            keyManagerFactory.init(keyStore, "mypassword".toCharArray());

            return new SSLSocketFactory(keyStore, "mypassword", trustStore);
        } catch (Exception e) {
            throw new AssertionError(e);
        }
    }
}

MainActivity

package com.example.https;

import android.app.Activity;
import android.os.AsyncTask;
import android.os.Bundle;
import android.view.View;
import android.widget.Button;
import android.widget.TextView;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringWriter;
import java.io.Writer;

import javax.net.ssl.SSLSocketFactory;

public class MainActivity extends Activity {

    private View loading;
    private TextView output;
    private Button button;

    SSLSocketFactory socketFactory = null;

    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        loading = findViewById(R.id.loading);
        output = (TextView) findViewById(R.id.output);
        button = (Button) findViewById(R.id.button);
        button.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                new CargaAsyncTask().execute(new Void[0]);
            }
        });
    }

    class CargaAsyncTask extends AsyncTask<Void, Void, String> {

        @Override
        protected void onPreExecute() {
            super.onPreExecute();
            loading.setVisibility(View.VISIBLE);
            button.setEnabled(false);
        }

        @Override
        protected String doInBackground(Void... params) {
            // Instantiate the custom HttpClient
            DefaultHttpClient client = new MyHttpClient(getApplicationContext());
            HttpGet get = new HttpGet("https://www.google.com");
            // Execute the GET call and obtain the response
            HttpResponse getResponse;
            String resultado = null;
            try {
                getResponse = client.execute(get);
                HttpEntity responseEntity = getResponse.getEntity();
                InputStream is = responseEntity.getContent();
                resultado = convertStreamToString(is);
            } catch (ClientProtocolException e) {
                e.printStackTrace();
            } catch (IOException e) {
                e.printStackTrace();
            }
            return resultado;
        }

        @Override
        protected void onPostExecute(String result) {
            super.onPostExecute(result);
            loading.setVisibility(View.GONE);
            button.setEnabled(true);
            if (result == null) {
                output.setText("Error");
            } else {
                output.setText(result);
            }
        }

    }

    public static String convertStreamToString(InputStream is) throws IOException {
        /*
         * To convert the InputStream to String we use the
         * Reader.read(char[] buffer) method. We iterate until the
         * Reader return -1 which means there's no more data to
         * read. We use the StringWriter class to produce the string.
         */
        if (is != null) {
            Writer writer = new StringWriter();

            char[] buffer = new char[1024];
            try {
                Reader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
                int n;
                while ((n = reader.read(buffer)) != -1) {
                    writer.write(buffer, 0, n);
                }
            } finally {
                is.close();
            }
            return writer.toString();
        } else {
            return "";
        }
    }
}

I hope it could be useful for someone else!! enjoy it!

Answer 2

I'm posting an updated answer since people still reference and vote on this question. I have had to change the socket factory code a few times as some things have changed since Android 4.0

// Trust manager / truststore
KeyStore trustStore=KeyStore.getInstance(KeyStore.getDefaultType());

// If we're on an OS version prior to Ice Cream Sandwich (4.0) then use the standard way to get the system
//   trustStore -- System.getProperty() else we need to use the special name to get the trustStore KeyStore
//   instance as they changed their trustStore implementation.
if (Build.VERSION.RELEASE.compareTo("4.0") < 0) {
    TrustManagerFactory trustManagerFactory=TrustManagerFactory
        .getInstance(TrustManagerFactory.getDefaultAlgorithm());
    FileInputStream trustStoreStream=new FileInputStream(System.getProperty("javax.net.ssl.trustStore"));
    trustStore.load(trustStoreStream, null);
    trustManagerFactory.init(trustStore);
    trustStoreStream.close();
} else {
    trustStore=KeyStore.getInstance("AndroidCAStore");
}

InputStream certificateStream=new FileInputStream(userCertFile);
KeyStore keyStore=KeyStore.getInstance("PKCS12");
try {
    keyStore.load(certificateStream, certPass.toCharArray());
    Enumeration<String> aliases=keyStore.aliases();
    while (aliases.hasMoreElements()) {
        String alias=aliases.nextElement();
        if (keyStore.getCertificate(alias).getType().equals("X.509")) {
            X509Certificate cert=(X509Certificate)keyStore.getCertificate(alias);
            if (new Date().after(cert.getNotAfter())) {
                // This certificate has expired
                return;
            }
        }
    }
} catch (IOException ioe) {
    // This occurs when there is an incorrect password for the certificate
    return;
} finally {
    certificateStream.close();
}

KeyManagerFactory keyManagerFactory=KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
keyManagerFactory.init(keyStore, certPass.toCharArray());

socketFactory=new SSLSocketFactory(keyStore, certPass, trustStore);

Hopefully this helps anyone still coming here in the future.

Answer 3

I think this is indeed the issue.

The first possibility, as far as I can tell, is that I need to configure this SSLSocketFactory with the devices' truststore that includes all of the standard Intermediate and endpoint Certificate Authorities

If this is true, how would I best go about loading this data?

Try something like this (you'll need to get your socket factory to use this default trust manager):

X509TrustManager manager = null;
FileInputStream fs = null;

TrustManagerFactory trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());

try
{
    fs = new FileInputStream(System.getProperty("javax.net.ssl.trustStore")); 
    keyStore.load(fs, null);
}
finally
{
    if (fs != null) { fs.close(); }
}

trustManagerFactory.init(keyStore);
TrustManager[] managers = trustManagerFactory.getTrustManagers();

for (TrustManager tm : managers)
{
    if (tm instanceof X509TrustManager) 
    {
        manager = (X509TrustManager) tm;
        break;
    }
}

EDIT: Please look at Pooks' answer before using the code here. It sounds like there's a better way to do this now.

Answer 4

Seems that you need to also set the hostname for your SSLSocketFactory.

Try adding the line

socketFactory.setHostnameVerifier(SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);

before creating a new connection with your SSLFactory.

Other than the differences in structures, we have similar code. In my implementation I just created my own extension of the DefaultHttpClient which looks similar to the majority of your code above. If this doesn't fix it I can post the working code for that and you can give that approach a try.

edit: here's my working version

    public class ActivateHttpClient extends DefaultHttpClient { 
    final Context context;


    /**
     * Public constructor taking two arguments for ActivateHttpClient.
     * @param context - Context referencing the calling Activity, for creation of
     * the socket factory.
     * @param params - HttpParams passed to this, specifically to set timeouts on the
     * connection.
     */
    public ActivateHttpClient(Context context, HttpParams params) {
        this.setParams(params);
    }


    /* (non-Javadoc)
     * @see org.apache.http.impl.client.DefaultHttpClient#createClientConnectionManager()
     * Create references for both http and https schemes, allowing us to attach our custom
     * SSLSocketFactory to either
     */
    @Override
    protected ClientConnectionManager createClientConnectionManager() {
        SchemeRegistry registry = new SchemeRegistry();
        registry.register(new Scheme("http", PlainSocketFactory
                .getSocketFactory(), 80));
        registry.register(new Scheme("https", newSslSocketFactory(), 443));
        return new SingleClientConnManager(getParams(), registry);
    }

    /**
     * Creation of new SSLSocketFactory, which imports a certificate from
     * a server which self-signs its own certificate.
     * @return
     */
    protected SSLSocketFactory newSslSocketFactory() {
        try {

            //Keystore must be in BKS (Bouncy Castle Keystore)
            KeyStore trusted = KeyStore.getInstance("BKS");

            //Reference to the Keystore
            InputStream in = context.getResources().openRawResource(
                    R.raw.cert);

            //Password to the keystore
            try {
                trusted.load(in, PASSWORD_HERE.toCharArray());
            } finally {
                in.close();
            }

            // Pass the keystore to the SSLSocketFactory. The factory is
            // responsible
            // for the verification of the server certificate.
            SSLSocketFactory sf = new SSLSocketFactory(trusted);

            // Hostname verification from certificate
            // http://hc.apache.org/httpcomponents-client-ga/tutorial/html/connmgmt.html#d4e506
            sf.setHostnameVerifier(SSLSocketFactory.STRICT_HOSTNAME_VERIFIER);
            return sf;

            // return new SSLSocketFactory(trusted);
        } catch (Exception e) {
            e.printStackTrace();
            throw new AssertionError(e);
        }
    }

}

and can be called as shown :

HttpParams params = new BasicHttpParams();

    // Set the timeout in milliseconds until a connection is established.
    int timeoutConnection = 500;
    HttpConnectionParams.setConnectionTimeout( params , timeoutConnection );

    // Set the default socket timeout (SO_TIMEOUT)
    // in milliseconds which is the timeout for waiting for data.
    int timeoutSocket = 1000;
    HttpConnectionParams.setSoTimeout( params , timeoutSocket );
            //ADD more connection options here!

    String url =
            "https:// URL STRING HERE";
    HttpGet get = new HttpGet( url );

    ActivateHttpClient client =
            new ActivateHttpClient( this.context, params );



    // Try to execute the HttpGet, throwing errors
    // if no response is received, or if there is
    // an error in the execution.
    HTTPResponse response = client.execute( get );

Answer 5

There's a simpler way to implement @jglouie 's solution. Basically, if you use a SSLContext and initialize it with null for the trust manager parameter, you should get a SSL context using the default trust manager. Note that this is not documented in the Android documentation, but the Java documentation for SSLContext.init says

Either of the first two parameters may be null in which case the installed security providers will be searched for the highest priority implementation of the appropriate factory.

Here's what the code would look like:

// This can be any protocol supported by your target devices.
// For example "TLSv1.2" is supported by the latest versions of Android
final String SSL_PROTOCOL = "TLS";

try {               
   sslContext = SSLContext.getInstance(SSL_PROTOCOL);

   // Initialize the context with your key manager and the default trust manager 
   // and randomness source
   sslContext.init(keyManagerFactory.getKeyManagers(), null, null);
} catch (NoSuchAlgorithmException e) {
   Log.e(TAG, "Specified SSL protocol not supported! Protocol=" + SSL_PROTOCOL);
   e.printStackTrace();
} catch (KeyManagementException e) {
   Log.e(TAG, "Error setting up the SSL context!");
   e.printStackTrace();
}

// Get the socket factory
socketFactory = sslContext.getSocketFactory();