Reputation: 2610
We are connecting to an https url using OpenJDK 13 and we have troubles with the SSL handshake. It always ends with javax.net.ssl.SSLHandshakeException: DH ServerKeyExchange does not comply to algorithm constraints
I have turned on the ssl debug output with system property -Djavax.net.debug=ssl:handshake
to see what is going on.
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "65 BC 66 59 53 A2 29 F5 DF 14 0F 9F 3D C2 AC E3 90 69 92 A2 53 4F 61 1E 1A BD AA 8A 67 4C 14 D9",
"session id" : "DB BA 01 32 DF 4A 86 36 71 FA DA 0D 9E D7 F3 3B 94 E9 32 84 95 2A 61 FA 8D 01 FB 87 75 E1 F8 A9",
"cipher suites" : "[TLS_AES_256_GCM_SHA384(0x1302), TLS_AES_128_GCM_SHA256(0x1301), TLS_CHACHA20_POLY1305_SHA256(0x1303), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA9), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCA8), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_RSA_WITH_CHACHA20_POLY1305_SHA256(0xCCAA), TLS_DHE_DSS_WITH_AES_256_GCM_SHA384(0x00A3), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_DHE_DSS_WITH_AES_128_GCM_SHA256(0x00A2), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_DSS_WITH_AES_256_CBC_SHA256(0x006A), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_DHE_DSS_WITH_AES_128_CBC_SHA256(0x0040), TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384(0xC02E), TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384(0xC032), TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256(0xC02D), TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256(0xC031), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384(0xC026), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384(0xC02A), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256(0xC025), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256(0xC029), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_DSS_WITH_AES_256_CBC_SHA(0x0038), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_DHE_DSS_WITH_AES_128_CBC_SHA(0x0032), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA(0xC005), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA(0xC00F), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA(0xC004), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA(0xC00E), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_EMPTY_RENEGOTIATION_INFO_SCSV(0x00FF)]",
"compression methods" : "00",
"extensions" : [
"server_name (0)": {
type=host_name (0), value=xxx
},
"status_request (5)": {
"certificate status type": ocsp
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
},
"supported_groups (10)": {
"versions": [x25519, secp256r1, secp384r1, secp521r1, x448, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"status_request_v2 (17)": {
"cert status request": {
"certificate status type": ocsp_multi
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
}
},
"extended_master_secret (23)": {
<empty>
},
"supported_versions (43)": {
"versions": [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1]
},
"psk_key_exchange_modes (45)": {
"ke_modes": [psk_dhe_ke]
},
"key_share (51)": {
"client_shares": [
{
"named group": x25519
"key_exchange": {
0000: 3C 66 CB BA 20 25 F8 AD 39 51 1E 7D 3F B7 22 0F <f.. %..9Q..?.".
0010: E6 DD 72 1A A8 29 57 3E E3 20 E8 20 80 00 8F 5F ..r..)W>. . ..._
}
},
]
}
]
}
"ServerHello": {
"server version" : "TLSv1",
"random" : "5E 69 ED 04 D0 13 E5 27 7C BC F1 A7 4F AA 29 49 88 15 C7 22 2B AE CA 3E 4A 34 F0 B4 F4 61 5C 2F",
"session id" : "5E 69 ED 04 B3 7D 7A 47 98 84 8E A5 C0 D8 9A CE 46 03 C0 CA C5 A9 23 9E CD 22 9C 1F E2 63 49 B7",
"cipher suite" : "TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033)",
"compression methods" : "00",
"extensions" : [
"renegotiation_info (65,281)": {
"renegotiated connection": [<no renegotiated connection>]
}
]
}
...Server certificate deleted...
"DH ServerKeyExchange": {
"parameters": {
"dh_p": {
0000: E9 E6 42 59 9D 35 5F 37 C9 7F FD 35 67 12 0B 8E ..BY.5_7...5g...
0010: 25 C9 CD 43 E9 27 B3 A9 67 0F BE C5 D8 90 14 19 %..C.'..g.......
0020: 22 D2 C3 B3 AD 24 80 09 37 99 86 9D 1E 84 6A AB "....$..7.....j.
0030: 49 FA B0 AD 26 D2 CE 6A 22 21 9D 47 0B CE 7D 77 I...&..j"!.G...w
0040: 7D 4A 21 FB E9 C2 70 B5 7F 60 70 02 F3 CE F8 39 .J!...p..`p....9
0050: 36 94 CF 45 EE 36 88 C1 1A 8C 56 AB 12 7A 3D AF 6..E.6....V..z=.
},
"dh_g": {
0000: 30 47 0A D5 A0 05 FB 14 CE 2D 9D CD 87 E3 8B C7 0G.......-......
0010: D1 B1 C5 FA CB AE CB E9 5F 19 0A A7 A3 1D 23 C4 ........_.....#.
0020: DB BC BE 06 17 45 44 40 1A 5B 2C 02 09 65 D8 C2 .....ED@.[,..e..
0030: BD 21 71 D3 66 84 45 77 1F 74 BA 08 4D 20 29 D8 .!q.f.Ew.t..M ).
0040: 3C 1C 15 85 47 F3 A9 F1 A2 71 5B E2 3D 51 AE 4D <...G....q[.=Q.M
0050: 3E 5A 1F 6A 70 64 F3 16 93 3A 34 6D 3F 52 92 52 >Z.jpd...:4m?R.R
},
"dh_Ys": {
0000: 65 6B 34 FD 17 66 20 CA 13 F4 7A 4C C6 0B 9C DE ek4..f ...zL....
0010: 0F 07 40 A2 95 6F D3 D1 91 86 5F ED 4F D4 AA 52 [email protected]...._.O..R
0020: 6F C0 31 84 02 8B AF F0 CE FA D2 92 D1 BA E8 99 o.1.............
0030: BF BF 80 AF 93 D1 B7 2C 45 36 94 14 0B FD 95 AA .......,E6......
0040: A6 07 52 26 60 0A CA 27 61 16 C9 5D 55 13 D9 9C ..R&`..'a..]U...
0050: 43 4B D5 47 AE 39 8B 2E A6 5A A2 25 86 11 34 7A CK.G.9...Z.%..4z
},
},
"signature": {
0000: 3E D4 F6 7C 57 EC CA CF FA DC 5C 18 01 E3 AF C2 >...W.....\.....
0010: 03 A0 94 58 51 9E DE 6B 5B 05 FB BD 9C A5 96 A5 ...XQ..k[.......
0020: F3 72 D0 A8 20 3A C7 F9 85 5E 2D EF 87 97 2F 38 .r.. :...^-.../8
0030: CD 0E 10 BD 0C FD 99 C3 96 A7 BD B8 40 E4 79 74 [email protected]
0040: 17 8A 6E DC B3 E7 8B 32 64 FD 97 4D 0E F0 F5 0C ..n....2d..M....
0050: C6 EC 86 44 83 DD A0 EB 8C 72 F1 70 DE 94 5D 74 ...D.....r.p..]t
0060: 97 E7 B6 AA B2 C0 9D 97 F8 CD DF 2B 55 33 A6 A4 ...........+U3..
0070: 54 87 AE AD 62 FF 21 34 68 C4 09 62 67 D1 4E 92 T...b.!4h..bg.N.
0080: BD D7 0E DC 86 31 3B D8 16 2A 19 70 7A C7 08 42 .....1;..*.pz..B
0090: 51 61 CC A5 E6 41 8E 56 8C 77 6A 88 39 51 7E C3 Qa...A.V.wj.9Q..
00A0: 6E E9 2F 74 65 A6 55 2B 0B 2A 58 DE C8 0C 7D 5D n./te.U+.*X....]
00B0: 85 06 4C 8C 53 EF EE 46 0D 20 00 7E 0A 59 7A 2E ..L.S..F. ...Yz.
00C0: 5F 97 F2 9C FC C2 7E 7C 6A 96 21 E5 C0 4F 53 0F _.......j.!..OS.
00D0: 89 42 7F 3A 02 1D 2C ED B9 B0 6C 37 D4 D8 79 58 .B.:..,...l7..yX
00E0: D9 F3 84 CE 82 67 B9 5D D1 76 BD 32 5D 37 6D 81 .....g.].v.2]7m.
00F0: C1 4D 19 D1 20 95 0C 20 A2 96 B0 BF DD 72 50 C8 .M.. .. .....rP.
}
}
I see two things here:
SSLHandshakeException
So I created a custom java.security file where I allow DH key size to be 768 bits. The property jdk.tls.disabledAlgorithms
is copied from the master java.security file and except the DH is exactly the same.
jdk.tls.disabledAlgorithms=SSLv3, RC4, DES, MD5withRSA, DH keySize < 768, EC keySize < 224, 3DES_EDE_CBC, anon, NULL
Now the SSL handshake works but the server set TLSv1.2 and TLS_AES_256_GCM_SHA384 cipher instead of TLS_DHE_RSA_WITH_AES_128_CBC_SHA which I don't understand.
"ClientHello": {
"client version" : "TLSv1.2",
"random" : "64 2E 04 AA 0D 6E 5D 53 D9 4A 5B B8 62 06 06 1C E3 54 BA 6F 7C 68 DE AC 3F B5 BD E1 7E 71 96 A6",
"session id" : "A4 A8 1A BD 44 39 6C 2C 9E 88 E3 72 B9 57 87 D7 CF 0C 7D E6 0F 31 B5 89 BE 61 C9 55 3D 22 E7 37",
"cipher suites" : "[TLS_AES_128_GCM_SHA256(0x1301), TLS_AES_256_GCM_SHA384(0x1302), TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384(0xC02C), TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256(0xC02B), TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384(0xC030), TLS_RSA_WITH_AES_256_GCM_SHA384(0x009D), TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384(0xC02E), TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384(0xC032), TLS_DHE_RSA_WITH_AES_256_GCM_SHA384(0x009F), TLS_DHE_DSS_WITH_AES_256_GCM_SHA384(0x00A3), TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256(0xC02F), TLS_RSA_WITH_AES_128_GCM_SHA256(0x009C), TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256(0xC02D), TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256(0xC031), TLS_DHE_RSA_WITH_AES_128_GCM_SHA256(0x009E), TLS_DHE_DSS_WITH_AES_128_GCM_SHA256(0x00A2), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384(0xC024), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384(0xC028), TLS_RSA_WITH_AES_256_CBC_SHA256(0x003D), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384(0xC026), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384(0xC02A), TLS_DHE_RSA_WITH_AES_256_CBC_SHA256(0x006B), TLS_DHE_DSS_WITH_AES_256_CBC_SHA256(0x006A), TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA(0xC00A), TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA(0xC014), TLS_RSA_WITH_AES_256_CBC_SHA(0x0035), TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA(0xC005), TLS_ECDH_RSA_WITH_AES_256_CBC_SHA(0xC00F), TLS_DHE_RSA_WITH_AES_256_CBC_SHA(0x0039), TLS_DHE_DSS_WITH_AES_256_CBC_SHA(0x0038), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256(0xC023), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256(0xC027), TLS_RSA_WITH_AES_128_CBC_SHA256(0x003C), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256(0xC025), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256(0xC029), TLS_DHE_RSA_WITH_AES_128_CBC_SHA256(0x0067), TLS_DHE_DSS_WITH_AES_128_CBC_SHA256(0x0040), TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA(0xC009), TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA(0xC013), TLS_RSA_WITH_AES_128_CBC_SHA(0x002F), TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA(0xC004), TLS_ECDH_RSA_WITH_AES_128_CBC_SHA(0xC00E), TLS_DHE_RSA_WITH_AES_128_CBC_SHA(0x0033), TLS_DHE_DSS_WITH_AES_128_CBC_SHA(0x0032)]",
"compression methods" : "00",
"extensions" : [
"server_name (0)": {
type=host_name (0), value=xxx
},
"status_request (5)": {
"certificate status type": ocsp
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
},
"supported_groups (10)": {
"versions": [secp256r1, secp384r1, secp521r1, sect283k1, sect283r1, sect409k1, sect409r1, sect571k1, sect571r1, secp256k1, ffdhe2048, ffdhe3072, ffdhe4096, ffdhe6144, ffdhe8192]
},
"ec_point_formats (11)": {
"formats": [uncompressed]
},
"signature_algorithms (13)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"signature_algorithms_cert (50)": {
"signature schemes": [ecdsa_secp256r1_sha256, ecdsa_secp384r1_sha384, ecdsa_secp521r1_sha512, rsa_pss_rsae_sha256, rsa_pss_rsae_sha384, rsa_pss_rsae_sha512, rsa_pss_pss_sha256, rsa_pss_pss_sha384, rsa_pss_pss_sha512, rsa_pkcs1_sha256, rsa_pkcs1_sha384, rsa_pkcs1_sha512, dsa_sha256, ecdsa_sha1, rsa_pkcs1_sha1, dsa_sha1]
},
"status_request_v2 (17)": {
"cert status request": {
"certificate status type": ocsp_multi
"OCSP status request": {
"responder_id": <empty>
"request extensions": {
<empty>
}
}
}
},
"extended_master_secret (23)": {
<empty>
},
"supported_versions (43)": {
"versions": [TLSv1.3, TLSv1.2, TLSv1.1, TLSv1]
},
"psk_key_exchange_modes (45)": {
"ke_modes": [psk_dhe_ke]
},
"key_share (51)": {
"client_shares": [
{
"named group": secp256r1
"key_exchange": {
0000: 04 70 89 F8 9D 7D 5E 07 DC 46 E3 57 5C 24 02 0A .p....^..F.W\$..
0010: C0 10 53 20 03 ED 94 F0 EC 07 64 BE 8C B7 93 D1 ..S ......d.....
0020: 19 97 11 81 24 A5 8F 93 46 B3 AE A6 F0 5F 3E E6 ....$...F...._>.
0030: EA 4B 11 B8 C5 45 D0 8E CD AF FB 3A BF 50 B0 5E .K...E.....:.P.^
0040: 56
}
},
]
},
"renegotiation_info (65,281)": {
"renegotiated connection": [<no renegotiated connection>]
}
]
}
"ServerHello": {
"server version" : "TLSv1.2",
"random" : "AD E4 31 7A 81 AF 56 73 AC 4F F0 8B 20 D7 6D 6B F9 53 94 97 03 5C 9F 1E B8 7D 15 01 CF 5D 41 A1",
"session id" : "A4 A8 1A BD 44 39 6C 2C 9E 88 E3 72 B9 57 87 D7 CF 0C 7D E6 0F 31 B5 89 BE 61 C9 55 3D 22 E7 37",
"cipher suite" : "TLS_AES_256_GCM_SHA384(0x1302)",
"compression methods" : "00",
"extensions" : [
"supported_versions (43)": {
"selected version": [TLSv1.3]
},
"key_share (51)": {
"server_share": {
"named group": secp256r1
"key_exchange": {
0000: 04 95 DD 75 15 26 7D 43 4D FF 75 DE 93 55 13 2D ...u.&.CM.u..U.-
0010: 2B 80 45 02 C1 12 11 9B 68 89 19 BC 2E 06 36 4B +.E.....h.....6K
0020: 07 44 86 54 F2 42 1F 0E 6E D6 13 1D 10 6E C8 61 .D.T.B..n....n.a
0030: DC E1 74 97 E1 29 3F 5A 2A FE 42 31 1B 85 C8 A9 ..t..)?Z*.B1....
0040: C4
}
},
}
]
}
Can anybody explain me how this change in jdk.tls.disabledAlgorithms
forces server to change TLS version and cipher?
Upvotes: 3
Views: 4309
Reputation: 38990
First, your second example is TLS1.3 not 1.2. Remember 1.3 still uses the version code for 1.2 (0x0303) in record header and Hello fixed fields, putting the real version only in extension 43, which you see in the ServerHello. Also the selected ciphersuite TLS_AES_256_GCM_SHA384(0x1302) is a 1.3-only ciphersuite; in fact all ciphersuite codes 0x13xx are 1.3-only.
There are lots of differences between the ClientHello in your first and second examples. The list of offered ciphersuites is different, the list of supported_groups (extension 10) is different, the offered keyshare is secp256r1 not x25519, and rfc5746 usaes extension versus SCSV.
Are you sure your second example was Java 13? I can almost match your first example with my Oracle build of 13.0.0 OOTB, the only unexpected difference being that I get additional EC curves in supported_groups, and that might be explained by your build or config using a different underlying provider. OTOH your second example much more closely matches my result for Java 11 again OOTB: the ciphersuites match, except I have rfc5746 SCSV while you have the extension, which IIRC can be due to a property setting; the groups and schemes match (except in 11.0.5) except mine misdecodes ecdsa_secp521r1 as ecdsa_secp512r1 which is obviously a bug (and is fixed in my 12 and 13).
However that doesn't explain the different server response, since both of them are valid TLS1.3 offers. The server seems to be reacting to the fact your second example, like my Java 11 (and 12), offers keyshare for secp256r1, while your first example, like my Java 13, offers x25519. Possibly the server dislikes x25519 so much it falls back, though falling back to 1.0 instead of 1.2 (which would still allow (FF)DHE with arbitrary non-rfc7919 group, and given this ClientHello also ECDHE with server-chosen secp256r1) is silly if not stupid. Of course the correct thing for a 1.3 server to do when it dislikes only the client keyshare(s) is to send HelloRetryRequest, which Java (11 up) would have honored and should have resulted in a 1.3 connection using secp256r1 and AES256GCM.
PS: If this server is publicly accessible, you might try https://www.ssllabs.com/ssltest on it and see what that finds.
Upvotes: 1