CODEWITHSUNDEEP
sslssl-certificatecloudflareamazon-eks
DmitrySemenov
DmitrySemenov

Reputation: 10305

cloudflare ssl for staging subdomain: sslv3 alert handshake failure

I have the following setup cloudflare -> aws nlb -> ingress nginx controller (aws eks) -> kubernetes service -> kubernetes pod.

Cloudflare has edge certificates enabled for *.project.com, project.com and are installed in ingress-nginx as

Cloudflare has origin server ssl cert for *.staging.project.com, *.project.com, project.com (3 hosts) that I installed inside kubernetes cluster.

extraArgs:
  default-ssl-certificate: ingress-nginx/cloudflare-origin-cert

However I'm unable to connect to argocd.staging.project.com via HTTPs due to handshake error. It should work as origin server cert has *.project.com and also *.staging.project.com.

Inside cloudflare I have just a single domain "project.com", as it seems cloudflare does not allow me to have a staging hosted zone.

What am I missing or doing wrong?

prod env works just fine with this setup, but not staging. I can change argocd.staging.project.com > /argocd-staging.project.com and everything would work, but I prefer to keep staging subdomain if possible.

DNS is working properly as in http call I get logs in ingress-nginx

✗  curl http://argocd.staging.project.com 
<html>
<head><title>308 Permanent Redirect</title></head>
<body>
<center><h1>308 Permanent Redirect</h1></center>
<hr><center>nginx</center>
</body>
</html>

but in curl https I don't see any logs inside ingress-nginx pod.

curl https://argocd.staging.project.com 
curl: (35) error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure

echo | openssl s_client -showcerts -servername argocd.staging.project.com -connect argocd.staging.project.com:443 2>/dev/null | openssl x509 -inform pem -noout -text
unable to load certificate
139926728525632:error:0909006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:745:Expecting: TRUSTED CERTIFICATE

cert info

openssl x509 -text -noout -in cloudflare-origin.cert 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0e:e8:98:22:e2:06:be:6d:18:ba:53:49:ef:ac:3a:ae:2b:a8:d3:e1
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = US, O = "CloudFlare, Inc.", OU = CloudFlare Origin SSL Certificate Authority, L = San Francisco, ST = California
        Validity
            Not Before: Dec 28 00:48:00 2021 GMT
            Not After : Dec 24 00:48:00 2036 GMT
        Subject: O = "CloudFlare, Inc.", OU = CloudFlare Origin CA, CN = CloudFlare Origin Certificate
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:c4:6e:4b:53:c7:bb:a3:7a:e4:52:79:39:20:c7:
                    67:1f:67:06:13:ad:8d:cf:48:ae:56:c0:ab:22:e7:
                    5f:22:1b:bb:35:24:74:62:1a:11:5e:be:c3:a7:70:
                    26:54:65:28:e5:bf:4c:d9:de:cc:1a:55:bf:e4:c4:
                    32:93:84:1f:7c:81:01:bb:20:74:72:e0:c9:f4:cc:
                    47:70:76:5e:e7:ce:43:cd:4f:5e:23:7b:b7:66:ac:
                    e6:ce:3a:1d:8f:1c:c1:5e:61:c2:da:64:46:6c:22:
                    00:4d:8a:97:ab:40:93:a8:dd:35:f0:26:43:a4:af:
                    25:5e:2f:27:d5:29:0a:e5:bf:c7:8f:79:8c:3d:07:
                    66:08:23:f9:a8:72:2b:e5:82:d9:90:a3:56:c5:4c:
                    be:a9:2a:12:90:e4:6c:0b:e4:12:45:9f:a9:e9:7c:
                    4b:66:36:3e:ff:f7:2b:a2:49:5d:6d:ef:7e:f4:3e:
                    5c:cf:7f:d2:70:e9:4f:06:c0:ca:ca:5f:ec:22:f7:
                    06:c0:0e:2d:f5:9f:b3:4c:0c:2f:b2:2e:fc:06:6a:
                    de:07:fa:cc:99:fa:83:35:a3:6d:48:13:da:23:2c:
                    52:9c:2f:30:0e:23:cc:af:e8:d1:31:cd:5d:95:bf:
                    cd:ba:43:91:06:c2:b4:b4:bc:ad:c2:e6:01:83:25:
                    d3:41
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Client Authentication, TLS Web Server Authentication
            X509v3 Basic Constraints: critical
                CA:FALSE
            X509v3 Subject Key Identifier: 
                13:86:11:20:22:E5:81:ED:B9:8A:5C:04:0F:3F:03:34:E1:86:55:0C
            X509v3 Authority Key Identifier: 
                keyid:24:E8:53:57:5D:7C:34:40:87:A9:EB:94:DB:BA:E1:16:78:FC:29:A4

            Authority Information Access: 
                OCSP - URI:http://ocsp.cloudflare.com/origin_ca

            X509v3 Subject Alternative Name: 
                DNS:*.staging.project.com, DNS:*.project.com, DNS:project.com
            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl.cloudflare.com/origin_ca.crl

    Signature Algorithm: sha256WithRSAEncryption
         63:fd:c0:b0:ad:95:e4:78:d2:d6:ae:62:8c:5d:a2:a6:c9:12:
         c0:56:02:2a:ba:04:fd:b7:74:d4:0d:ad:5e:55:78:67:63:1a:
         79:83:58:91:b4:a9:77:e1:5e:5d:86:ad:e2:5b:03:a1:88:ff:
         88:bb:f4:29:7d:83:96:89:f8:44:a4:4e:79:c3:ab:14:89:15:
         ea:af:a5:66:f4:6a:fe:2a:a5:55:de:0f:36:a5:cb:95:59:ee:
         3a:51:6b:d3:ca:3c:0a:bc:66:60:ff:77:81:91:57:91:3a:a5:
         ea:05:30:aa:69:01:95:48:44:04:e8:78:a7:bf:03:9b:7e:65:
         f7:5d:91:5d:a9:a2:67:5a:3c:c8:7f:9e:4e:3f:3a:2a:2a:5a:
         68:4b:b5:e2:a1:68:a1:ff:6d:d4:39:9d:00:ab:89:c7:34:aa:
         5b:87:fe:ba:61:c2:94:51:5d:59:c5:a0:0a:dc:0c:23:24:19:
         bc:37:ad:1f:8c:bd:71:89:63:b2:a8:a3:24:20:fc:dd:0f:d9:
         15:b4:a2:b8:8f:7a:c6:a6:50:20:a0:fd:de:1a:79:c6:30:86:
         79:bf:ea:46:e3:1b:e6:86:3b:89:67:d2:c5:bf:d8:62:9f:52:
         6c:d2:1f:b5:f6:03:56:2b:23:5e:30:7a:3e:78:39:f7:cd:a0:
         d0:3c:da:69

However for production environment (staging omitted in URL) everything works and handshake is normal.

echo | openssl s_client -showcerts -servername argocd.project.com -connect argocd.project.com:443 2>/dev/null | openssl x509 -inform pem -noout -text  
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0e:2d:db:f3:59:21:a2:91:e4:67:79:17:ff:71:8d:e5
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
        Validity
            Not Before: Jun 15 00:00:00 2021 GMT
            Not After : Jun 14 23:59:59 2022 GMT
        Subject: C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:8d:99:4f:55:aa:0c:c2:4d:1b:57:23:e8:73:09:
                    7f:de:d4:ae:50:f8:19:74:0a:23:0f:cc:3e:64:c1:
                    bf:66:56:72:06:4a:c5:0c:13:1f:43:b9:d5:f9:88:
                    e6:f5:4c:4a:02:ee:76:37:9d:ee:e6:26:7d:be:3e:
                    fc:42:a5:97:20
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        X509v3 extensions:
            X509v3 Authority Key Identifier: 
                keyid:A5:CE:37:EA:EB:B0:75:0E:94:67:88:B4:45:FA:D9:24:10:87:96:1F

            X509v3 Subject Key Identifier: 
                FA:15:4F:CE:7F:3D:C9:27:5A:D3:87:C1:ED:68:A9:FC:CC:BC:E2:84
            X509v3 Subject Alternative Name: 
                DNS:*.project.com, DNS:sni.cloudflaressl.com, DNS:project.com
            X509v3 Key Usage: critical
                Digital Signature
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 CRL Distribution Points: 

                Full Name:
                  URI:http://crl3.digicert.com/CloudflareIncECCCA-3.crl

                Full Name:
                  URI:http://crl4.digicert.com/CloudflareIncECCCA-3.crl

            X509v3 Certificate Policies: 
                Policy: 2.23.140.1.2.2
                  CPS: http://www.digicert.com/CPS

            Authority Information Access: 
                OCSP - URI:http://ocsp.digicert.com
                CA Issuers - URI:http://cacerts.digicert.com/CloudflareIncECCCA-3.crt

            X509v3 Basic Constraints: critical
                CA:FALSE
            CT Precertificate SCTs: 
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 46:A5:55:EB:75:FA:91:20:30:B5:A2:89:69:F4:F3:7D:
                                11:2C:41:74:BE:FD:49:B8:85:AB:F2:FC:70:FE:6D:47
                    Timestamp : Jun 15 16:30:55.567 2021 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:46:02:21:00:DD:C3:A2:FE:62:CE:34:30:BF:41:A3:
                                3D:E3:D3:4B:7A:0C:DD:BF:1E:A0:81:B0:5B:63:0E:A3:
                                83:6B:5D:AF:5C:02:21:00:C7:5C:0F:71:C9:61:11:5A:
                                A8:2F:5F:9A:31:A4:2A:C0:83:B6:2A:29:FC:BD:5D:FA:
                                3C:CF:B5:F6:1E:EE:F0:6B
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 22:45:45:07:59:55:24:56:96:3F:A1:2F:F1:F7:6D:86:
                                E0:23:26:63:AD:C0:4B:7F:5D:C6:83:5C:6E:E2:0F:02
                    Timestamp : Jun 15 16:30:55.564 2021 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:20:25:E2:6B:36:61:E9:F4:EC:28:DE:1D:E3:
                                18:6F:E2:0A:03:EF:29:45:F3:09:0B:27:45:6F:51:78:
                                D5:3A:2A:83:02:21:00:A4:34:A0:B5:D5:FD:F2:42:13:
                                31:93:DF:C4:AD:3E:A7:48:C6:69:C1:9D:04:7A:EA:C7:
                                27:6E:88:69:9B:B9:BF
                Signed Certificate Timestamp:
                    Version   : v1 (0x0)
                    Log ID    : 51:A3:B0:F5:FD:01:79:9C:56:6D:B8:37:78:8F:0C:A4:
                                7A:CC:1B:27:CB:F7:9E:88:42:9A:0D:FE:D4:8B:05:E5
                    Timestamp : Jun 15 16:30:55.627 2021 GMT
                    Extensions: none
                    Signature : ecdsa-with-SHA256
                                30:45:02:21:00:FA:13:20:B1:07:70:46:F4:C2:AD:F0:
                                1C:10:A7:8D:92:23:2C:8A:34:E0:1C:7F:59:8A:CB:7B:
                                C2:CF:07:95:37:02:20:50:78:FA:DF:8D:A4:9C:B9:73:
                                1F:18:ED:51:06:33:8D:B4:F6:CC:0D:8D:46:69:CB:AB:
                                93:17:D2:64:1F:2D:B3
    Signature Algorithm: ecdsa-with-SHA256
         30:46:02:21:00:fc:1b:7b:6f:de:f2:29:5a:11:0c:92:f8:05:
         31:1b:7c:68:f7:6e:e4:0b:5d:15:67:dd:f4:c9:00:d7:77:ad:
         46:02:21:00:a0:98:25:6a:19:3b:ac:51:68:f5:de:9d:cc:93:
         22:b2:ca:18:c8:e9:ec:06:79:77:01:ba:fb:3a:41:3d:2d:cd

Upvotes: 0

Views: 4156

Answers (1)

DmitrySemenov
DmitrySemenov

Reputation: 10305

Ok found - it's the limitation of universal cloudflare certificate that doesn't cover subdomains :(

from their docs:

Only some of your subdomains return SSL errors

Symptom Cloudflare Universal SSL and regular Dedicated SSL certificates only cover the root-level domain (example.com) and one level of subdomains (*.example.com). If visitors to your domain observe errors accessing a second level of subdomains in their browser (such as dev.www.example.com) but not the first level of subdomains (such as www.example.com), resolve the issue using one of the following methods below.

Resolution

  • Ensure the domain is at least on a Business plan and upload a Custom SSL certificate that covers dev.www.example.com, or
  • purchase a Dedicated SSL certificate with Custom Hostnames that covers dev.www.example.com, or
  • if you have a valid certificate for the second level subdomains at your origin web server, click the orange cloud icon beside the dev.www hostname in the Cloudflare DNS app for example.com.

See here: https://support.cloudflare.com/hc/en-us/articles/200170566-Troubleshooting-SSL-errors#h_55e4d315-c60d-4798-9c4c-c75d9baed1b7

Upvotes: 4

Related Questions