https SSL cert verification with Nginx fails with custom built libcurl+openssl, but works with system curl

Question

I'm using libcurl to talk to two separate web servers over https.

The first server is running Lighttpd 1.4.26-1 on 64-bit Ubuntu 10.04. Lightty is built against openssl version 0.9.8k.

The second server is running Nginx 0.7.65 on 32-bit Ubuntu 10.04. Nginx is built against openssl version 0.9.8k.

Both of these servers use the same certificate, and DNS is round-robined between them. I use /etc/hosts to manage which of the two I'm hitting.

I'm using libcurl version 7.21.2 built against openssl 1.0.0c.

With this, libcurl can successfully make an https request to the lightty server, but not the nginx server. I can duplicate this with the curl command-line interface. Here's the output of curl -v:

* About to connect() to <hostname> port 443 (#0)
*   Trying <ip>... connected
* Connected to <hostname> (<ip>) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: cadata
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS alert, Server hello (2):
* SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
* Closing connection #0
curl: (60) SSL certificate problem, verify that the CA cert is OK. Details:
error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

If I use an older version of curl (7.19.7 with openssl 0.9.8l) that I happen to have on the same system, the same request works:

* About to connect() to <hostname> port 443 (#0)
*   Trying <ip>... connected
* Connected to <hostname> (<ip>) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: certdata
  CApath: none
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using DHE-RSA-AES256-SHA
* Server certificate:

Thinking this was something to do with the newer versions of curl and/or openssl, I rebuilt my curl and libcurl with the same version and options enabled:

Broken (my build):
seldon:bin ryanowen$ ./curl -V
curl 7.19.7 (i386-apple-darwin10.7.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps 
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz 

Works (system version):
seldon:bin ryanowen$ /usr/bin/curl -V
curl 7.19.7 (universal-apple-darwin10.0) libcurl/7.19.7 OpenSSL/0.9.8l zlib/1.2.3
Protocols: tftp ftp telnet dict ldap http file https ftps 
Features: GSS-Negotiate IPv6 Largefile NTLM SSL libz 

Unfortunately, this didn't seem to make a difference. My curl build still fails (but only when talking to the nginx server) while the system build of course still works.

What could be different about my libcurl build that is causing it to fail the SSL handshake? Are there any options not reflected in the -V output that I need to enable? Anything about the Nginx side of things that I should look into?

Answer 1

I finally discovered the problem on the Nginx side. The Nginx config referenced the server certificate and key, but not the CA bundle. Concatenating the CA bundle onto the end of the server certificate file seems to have solved the problem.

The CA data I was giving to libcurl included the CA certificate, but not some of the other certificates in the chain. I guess the system curl must have had those certificates stashed away somewhere.