SAN getting cleared in SSL certificate

Question

I'm trying to setup SSL connection between Kafka (0.10.0v) and Filebeat (5.6.0v). I've done the below.

1. By following this document I've setup SSL communication between brokers. For now, client authentication is not required.
2. Provided the CA used for signing Kafka server certificates in filebeat.yml, so that the communication between Kafka and Filebeat is encrypted.

But on starting filebeat service, I'm getting the below error.

2018/07/06 17:22:01.128453 log.go:12: WARN Failed to connect to broker xx.xx.xxx:9093: x509: cannot validate certificate for xx.xx.xxx.114 because it doesn't contain any IP SANs
2018/07/06 17:22:01.128488 log.go:16: WARN kafka message: client/metadata got error from broker while fetching metadata:%!(EXTRA x509.HostnameError=x509: cannot validate certificate for xx.xx.xxx.114 because it doesn't contain any IP SANs)
2018/07/06 17:22:01.128507 log.go:12: WARN client/metadata fetching metadata for all topics from broker xx.xx.xxx.115:9093
2018/07/06 17:22:01.142781 log.go:12: WARN Failed to connect to broker xx.xx.xxx.115:9093: x509: cannot validate certificate for xx.xx.xxx.115 because it doesn't contain any IP SANs
2018/07/06 17:22:01.142815 log.go:16: WARN kafka message: client/metadata got error from broker while fetching metadata:%!(EXTRA x509.HostnameError=x509: cannot validate certificate for xx.xx.xxx.115 because it doesn't contain any IP SANs)

On Checking the server certificate before signing with CA, I was able to see the SAN (IP) being set as shown in below

openssl req -noout -text -in cert-file Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject: C=XX, ST=XX, L=XX, O=XXXX, OU=XXX, CN=*
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:9d:e3:94:be:33:d8:52:48:64:f6:db:5a:09:23:
                    22:64:b0:e2:75:14:2b:a2:9c:1e:43:6d:6a:d2:aa:
                    ff:84:46:ba:50:c1:57:4b:5f:2f:06:6b:ff:89:5a:
                    24:73:dd:7b:45:29:3f:74:1b:11:e3:53:93:bf:99:
                    02:8f:dc:95:7c:4e:3c:cb:67:8b:fe:e2:97:2f:0f:
                    45:92:9f:9f:03:76:e8:5b:16:93:8b:6c:b1:78:18:
                    63:e8:ec:1c:84:98:64:13:e4:12:eb:b7:9a:9b:93:
                    02:06:41:c7:d2:21:65:7d:9a:68:e4:8c:ec:19:47:
                    b8:47:a6:6c:04:93:0e:f4:04:b0:d4:1b:c4:9c:92:
                    d5:da:50:17:a6:e8:5a:bd:6c:7e:8b:bb:08:67:48:
                    ef:59:14:4c:8a:c6:4e:e7:ac:c1:eb:d0:60:56:dd:
                    af:54:7d:d9:35:ed:26:cc:ee:e2:8a:5d:18:0e:86:
                    d7:ba:13:b7:bb:e2:54:8f:14:a1:d1:25:ea:1b:e7:
                    ed:38:fb:d9:e6:f4:7d:b7:ef:ea:b1:18:39:35:d1:
                    53:bf:59:b2:2a:33:e5:23:38:16:04:bc:54:da:63:
                    0e:35:de:a2:41:5e:72:e7:4a:ea:24:3b:52:c1:61:
                    b3:82:32:e7:0c:cd:02:fd:11:93:15:79:76:46:b7:
                    17:bb
                Exponent: 65537 (0x10001)
        Attributes:
        Requested Extensions:
            X509v3 Subject Alternative Name:
                IP Address:xx.xx.xxx.115
            X509v3 Subject Key Identifier:
                9A:41:EC:4C:FA:D5:3D:C6:F8:18:A7:24:FB:5C:EA:03:70:C2:FC:71
    Signature Algorithm: sha256WithRSAEncryption
         1d:61:c2:84:21:f7:ac:05:9c:83:2f:52:b2:76:ac:4a:b6:79:
         41:b8:e6:35:c2:92:bb:a4:8f:83:04:39:63:c4:3b:99:96:a4:
         4a:89:f8:23:49:d4:da:82:2d:cc:2e:fc:5e:16:f8:ed:95:d2:
         7a:09:e4:42:a3:da:74:f2:da:48:37:06:75:d5:56:36:28:59:
         d6:9c:d0:e3:1d:f9:e4:46:e2:e5:0d:05:19:ab:de:72:dc:68:
         d3:6d:3d:a3:59:9e:b4:6b:37:69:e6:cd:17:08:bb:44:09:06:
         f3:c3:66:44:94:93:c2:54:4b:f8:ae:eb:7e:11:a9:8c:f6:b4:
         07:da:9c:4b:f1:fa:ee:24:cf:ae:c1:aa:e4:82:03:4d:30:d3:
         28:1a:2f:84:64:61:bc:27:da:47:81:0c:05:a4:ea:36:61:74:
         7b:6c:d9:31:81:7f:fa:7c:a9:02:5b:5c:ef:6d:95:84:59:f6:
         cc:84:2c:81:25:7a:ef:dc:99:4c:78:c4:b4:18:43:b4:a5:18:
         cc:63:75:ba:76:ef:96:7b:63:f9:7d:30:4a:3f:cc:f2:6a:ea:
         12:de:da:ab:a0:2d:42:a2:a1:64:24:5b:c4:b9:51:e6:14:8d:
         a1:1a:d6:bb:11:2c:23:cc:2d:6f:ca:4e:3e:11:ee:74:3a:2e:
         9c:da:fd:ba

To check the ssl connection I ran the below command and got the output shown beneath the command

openssl s_client -showcerts -connect XX.XX.XX.115:9093

CONNECTED(00000003)
depth=0 C = XX, ST = XX, L = XX, O = XXXX, OU = XX, CN = *
verify error:num=18:self signed certificate
verify return:1
depth=0 C = XX, ST = XX, L = XX, O = XXXX, OU = XX, CN = *
verify return:1
Certificate chain
0 s:/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
i:/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
-----BEGIN CERTIFICATE-----
MIIDJDCCAgwCCQC13GlMA2vKPzANBgkqhkiG9w0BAQUFADBUMQswCQYDVQQGEwJT
VzELMAkGA1UECAwCS0ExCzAJBgNVBAcMAktBMREwDwYDVQQKDAhFcmljc3NvbjEM
MAoGA1UECwwDQ1JTMQowCAYDVQQDDAEqMB4XDTE4MDcwNjE3MTYyN1oXDTI4MDcw
MzE3MTYyN1owVDELMAkGA1UEBhMCU1cxCzAJBgNVBAgTAktBMQswCQYDVQQHEwJL
QTERMA8GA1UEChMIRXJpY3Nzb24xDDAKBgNVBAsTA0NSUzEKMAgGA1UEAwwBKjCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMO5YHnHocbmN+zg/Qqq+aUJ
vQ9w1lTMfP6BHuobG2xd20hoXsj+DpdDrBry+iF1MvuOC+xYl25ODb7WhllVO+bz
kwPh1bbjGe7+PGKu3cLIAK9WWnlt3KCx0UsUhF7HuG9YcbpNz+xxjb6wlH0q4cre
QT9Q4aNhLn67HUA/ZjEXA9OEzxyiqEctYPGZIpkcn98jmymS+aEIBkiWGUS45+Cj
fg4jqy6Ow7vmC/3qndQ0iU4zxZgGkjhLwc7a30CNfQa3jBHj24ajOzAVb1US/hCZ
X2Y1QOVDLa6hfMkEXUivoD60nbcGLajS3WmJIer4FP/FZKACVoclrVkTELFI3GMC
AwEAATANBgkqhkiG9w0BAQUFAAOCAQEAr0Pnf/UHjVfWQWfSlSP4DQIoL5LesFY+
qRqY9db4j0Msg1q91zRoUiioe7w5Vw5Nd78bwwLpBFmFqbKga7ymid42+ZMglc/d
/laiv1TxbCdEQjnLIxOtQJ7gFysxV+XwKsCqUSQHYaTjOibuPR2LbbzTCO17PRMy
b+vuhD6+WBlpefBArm+3HildWQz7qn5Zt/PB1oANU0HwkMOyDa9dpoTiM2yjndsK
1Y1mnfRLm/+a9Z9q/VGwNBQPBT/xI/QgaGtE1k/3gJUn/vOuJ6lLM4D/b3wPcI+J
KKhWpDEH3rUKyoQTpWHeN4x+a3P0iKl7B06cv+ONpCoa8HutL5hv0w==
-----END CERTIFICATE-----
1 s:/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
i:/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
-----BEGIN CERTIFICATE-----
MIIDezCCAmOgAwIBAgIJAOeDYHjSIpe4MA0GCSqGSIb3DQEBCwUAMFQxCzAJBgNV
BAYTAlNXMQswCQYDVQQIDAJLQTELMAkGA1UEBwwCS0ExETAPBgNVBAoMCEVyaWNz
c29uMQwwCgYDVQQLDANDUlMxCjAIBgNVBAMMASowHhcNMTgwNzA2MTcxNjE1WhcN
MjgwNzAzMTcxNjE1WjBUMQswCQYDVQQGEwJTVzELMAkGA1UECAwCS0ExCzAJBgNV
BAcMAktBMREwDwYDVQQKDAhFcmljc3NvbjEMMAoGA1UECwwDQ1JTMQowCAYDVQQD
DAEqMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA5mey7Asqpk+muzH5
et7xtGW+3hZlZDrv6FWd5GneFCucF/GGjY3yTEqlHa5wxth8HIUNYcCS2tQjSsmv
jxAlhE7lF5R/iUgYvlC+LTvCP2q/g1us5TK+eCwVhQAL4QAf9igytvBE/Fsnn6fX
AM1EkPDbnUP1I+fPPHgFRdF6geRVTEUc1SR/LOAZbLzahGQ5WMt53GRj5xs08WEn
zD9eWeZPKTSP97q4jhG9JFkeZwFXFN5Xzif+podLIzM0LTBD04xSuIsH+A6XT+Bf
LS3QKjDs0PART9po+pSZKdI3Gv+gLGgUOjkwOAFyJMCi0g6Z5JsxmsRD8ae+WScg
CS64vwIDAQABo1AwTjAdBgNVHQ4EFgQUgs1yRIV0x6s3dZkIAGbATjFjDCgwHwYD
VR0jBBgwFoAUgs1yRIV0x6s3dZkIAGbATjFjDCgwDAYDVR0TBAUwAwEB/zANBgkq
hkiG9w0BAQsFAAOCAQEAErCcrBhHEEm7aNQpTnjTZd8I/3oGAju2bdw1ysqYKZLQ
spVFU20gvHAshMCg/+fTBcKVwDkzsjP1TDeuQ81SOyWHWNtNGsuVdsJN8oIxx5AG
Llv0WkedazvxG3GamxK8vtiitjgwDdS/K4cT5HTredCiGEbD5Qft2LYleMpqGzRo
84xICYEgSZpHEY29QXqcM0kWiz6jo+TGtjRtCHBeawdfQb2OuzexkOoKOadj4liH
8DzGIoFE2RBbjc7kd6idUgxh9Uenra/ks7uhf6IFjztAvGMPPEY9qt0CPR+cPYIn
kykBKeYZ1MsPxYlrQGqeYcnz6bM/6G58JIjTAOitlQ==
-----END CERTIFICATE-----
Server certificate
subject=/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
issuer=/C=XX/ST=XX/L=XX/O=XXXX/OU=XX/CN=*
No client certificate CA names sent
Server Temp Key: ECDH, secp521r1, 521 bits
SSL handshake has read 2258 bytes and written 441 bytes
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 5B3FA65DDE9A09886C1A725F46758274B810610F1DF11D23811773D44362A7F3
Session-ID-ctx:
Master-Key: 8105A8F49419A1D6AB3C06810FB3CCCF0A668DC7F812A9D5B2379AE7BAC4BEC0270A47C68E8A1B4549845E1B49CD2BF8
Key-Arg : None
Krb5 Principal: None
PSK identity: None
PSK identity hint: None
Start Time: 1530898013
Timeout : 300 (sec)
Verify return code: 18 (self signed certificate)

Can someone please help me to understand where I could be going wrong.

Answer 1

As @Steffen highlighted the SAN was not set by the CA while signing the CSR. By following this link I was able to set SAN in the signed certificate. Thanks a lot for your suggestions!

Answer 2

You've created a certificate with a CN of *, probably because you think that this does match everything. Only, it doesn't. A * matches only a single label of a domain name. And it does not match an IP address at all. And use of CN is obsolete for years anyway and instead Subject Alternative Names (SAN) should be used.

... x509: cannot validate certificate for xx.xx.xxx.115 because it doesn't contain any IP SANs

It looks like you specify the target of the connection with an IP address. In this case the certificate should have a SAN of type iPAddress with the value of the specific IP address. Only, your certificate does not have such SAN, in fact it has no SAN at all.

   Requested Extensions:
        X509v3 Subject Alternative Name:
            IP Address:xx.xx.xxx.115

It looks like that your CSR has included a SAN for this specific IP address. Only, whoever signed the certificate did not include this extension into the certificate as can be seen from doing openssl x509 -text ... on the certificate shown in the openssl s_client ... output. In case you've created the certificate yourself see the various questions on how to no loose the SAN in the CSR when creating the certificate.