How can I fetch an SSL page using Diffie-Hellman with a 480-bit key length?

Question

I'm trying to fetch page that is using DH with a 480-bit key length but I cannot do so using the Apache HTTP Client. I can successfully fetch the page if I use Async HTTP Client with the JDK Provider.

I know that the JDK has limitations around the max size but this issue seems to be the other way around. The key size is too small and not a multiple of 64.

I've written a SSCCE to illustrate this behaviour and after two days of hunting, I'm at my wit's end.

import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;

import javax.net.ssl.SSLContext;

import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.config.Registry;
import org.apache.http.config.RegistryBuilder;
import org.apache.http.conn.HttpClientConnectionManager;
import org.apache.http.conn.socket.ConnectionSocketFactory;
import org.apache.http.conn.socket.PlainConnectionSocketFactory;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.conn.BasicHttpClientConnectionManager;
import org.apache.http.util.EntityUtils;
import org.junit.Assert;
import org.junit.Test;

import com.ning.http.client.AsyncHttpClient;
import com.ning.http.client.AsyncHttpClientConfig;
import com.ning.http.client.providers.jdk.JDKAsyncHttpProvider;

public class DH480Test extends Assert {

    private final String URL = "https://dh480.badssl.com/";

    @Test
    public void testNingHTTPClientWithJDKProvider() {
        AsyncHttpClientConfig config = new AsyncHttpClientConfig.Builder().build();
        AsyncHttpClient client = new AsyncHttpClient(new JDKAsyncHttpProvider(config), config);

        try {
            client.prepareGet(URL).execute().get();
        } catch (Exception e) {
            e.printStackTrace();
            fail();
        }
    }

    @Test
    public void testApacheHTTClient() throws NoSuchAlgorithmException, KeyManagementException {
        SSLContext sslContext = SSLContext.getInstance("SSL");
        sslContext.init(null, null, null);

        Registry<ConnectionSocketFactory> registry = RegistryBuilder.<ConnectionSocketFactory>create()
                .register("https", new SSLConnectionSocketFactory(sslContext))
                .register("http", new PlainConnectionSocketFactory())
                .build();
        HttpClientConnectionManager ccm = new BasicHttpClientConnectionManager(registry);
        HttpClientBuilder builder = HttpClientBuilder.create();
        builder.setConnectionManager(ccm);

        HttpClient client = builder.build();

        try {
            HttpGet get = new HttpGet(URL);
            HttpResponse r = client.execute(get);
            EntityUtils.consume(r.getEntity());
        } catch (Exception e) {
            e.printStackTrace();
            fail();
        }
    }
}

Here's the stack trace of the the failing test.

javax.net.ssl.SSLException: java.lang.RuntimeException: Could not generate DH keypair
    at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949)
    at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1906)
    at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1889)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1410)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:290)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:259)
    at org.apache.http.impl.conn.HttpClientConnectionOperator.connect(HttpClientConnectionOperator.java:125)
    at org.apache.http.impl.conn.BasicHttpClientConnectionManager.connect(BasicHttpClientConnectionManager.java:318)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:363)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:219)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
    at DH480Test.testApacheHTTClient(DH480Test.java:59)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:234)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:74)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
Caused by: java.lang.RuntimeException: Could not generate DH keypair
    at sun.security.ssl.DHCrypt.<init>(DHCrypt.java:142)
    at sun.security.ssl.DHCrypt.<init>(DHCrypt.java:114)
    at sun.security.ssl.ClientHandshaker.serverKeyExchange(ClientHandshaker.java:708)
    at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:268)
    at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979)
    at sun.security.ssl.Handshaker.process_record(Handshaker.java:914)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062)
    at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
    at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
    ... 41 more
Caused by: java.security.InvalidAlgorithmParameterException: Prime size must be multiple of 64, and can only range from 512 to 2048 (inclusive)
    at com.sun.crypto.provider.DHKeyPairGenerator.initialize(DHKeyPairGenerator.java:120)
    at java.security.KeyPairGenerator$Delegate.initialize(KeyPairGenerator.java:674)
    at sun.security.ssl.DHCrypt.<init>(DHCrypt.java:128)
    ... 49 more

java.lang.AssertionError
    ...

---

Here's another standalone test that showcases that I am able to fetch the page using the default JDK networking libs. This is the underlying networking implementation used by the async http client. Here are my JVM details

➜  ~  java -version
java version "1.8.0_66"
Java(TM) SE Runtime Environment (build 1.8.0_66-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode)

@Test
public void testHTTPURLConnection() throws IOException {
    try {
        java.net.URL url = new URL(URL);
        HttpsURLConnection connection = (HttpsURLConnection) url.openConnection();
        connection.setHostnameVerifier((s, sslSession) -> true);
        connection.setRequestMethod("GET");
        connection.setReadTimeout(15 * 1000);
        connection.connect();
        System.out.println(IOUtils.toString(connection.getInputStream()));
    } catch (Exception e) {
        fail();
    }
}

Answer 1

Disclaimer: I'm the maintainer of AsyncHttpClient (please, don't call it Ning).

Fun fact: this does work with AHC's JDK provider when running with JDK8, but not with JDK7!

Now: AHC JDK provider (which is a mere toy that is going away in AHC2, you shouldn't use it) boils down to Http(s)UrlConnection, so it's expected to get the same results.

Then, NIO based clients such as the other AHC providers, and Apache HttpComponents can't use the underlying HTTPS components that are being used by HttpsUrlConnection, because they are not public!

So they have to directly use javax.net.ssl.SSLEngine. But, this API was intended for generic TLS, not for the protocols built on top of it such as HTTPS.

For example, before JDK7, this cause a huge security issue on all Java NIO based HTTP clients, because they didn't have a way to properly perform hostname verification. Hostname verification is specific to HTTPS, so you can pass a HostnameVerifier to HttpsUrlConnection (HTTPS specific), but not to SSLEngine (generic TLS).

Now, since JDK7, we can finally configure a protocol (HTTPS or LDAP) on the SSLEngine so it takes care of the HTTPS specific behavior. But still, you have way less control than with HttpsUrlConnection.

Here, it seems that you found a JDK bug, and that HttpsUrlConnection and SSLEngine w/ HTTPS protocol enabled don't behave the same way.

For example, whatever I do (whatever cyphersuites I enable), I can't manage to get the BadSSL.com server pick TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, which is the one HttpsUrlConnection picks. It always pick TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 instead (no elliptic curve). See ServerHello when enabling TLS debug (-Djavax.net.debug=all).

Answer 2

I am not an expert and never worked with DH in java but after seeing your logs and little exploration of the code at DHKeyPairGenerator Grepcode it is really very clear in both the initialize function that it will always throw exception if Prime size < 512 as it is in written in code,

83         if ((keysize < 512) || (keysize > 1024) || (keysize % 64 != 0)) {
84             throw new InvalidParameterException("Keysize must be multiple "
85                                                 + "of 64, and can only range "
86                                                 + "from 512 to 1024 "
87                                                 + "(inclusive)");
88         }

and

118    if ((pSize < 512) || (pSize > 1024) ||
119            (pSize % 64 != 0)) {
120            throw new InvalidAlgorithmParameterException
121                ("Prime size must be multiple of 64, and can only range "
122                 + "from 512 to 1024 (inclusive)");
123        }

So it's for sure that you can not use these standard libraries for 480-bit key length.

It might be available in some other libraries which you need to explore, we used bouncycastle libraries to do some encryption stuff that might have those capabilities.

I little researched and found the same DHKeyPairGenerator class in bouncycastle package with no constraints and checks on key size.