TLS 1.2 - key-pair size and encryption key length

Question

Assuming a (java based) server that uses java 1.8 and a (java based) client that uses java 8 as well. And given that java 8 supports by default TLS 1.2. Is it guaranteed that a 512 bit encryption will be used? if not what affects the symmetric key size?

Another question about the RSA key (1024 or 2048). Is RSA key part of the certificate or the RSA key is determined during the TLS handshake?

Many thanks Guy

Answer 1

SSL Labs has a github page where they've been making suggestions about stuff like this since 2013: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices

The specific answer to your question is here: https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#11-use-2048-bit-private-keys

I pasted it below, but know that my answer may go out of date while their document continues to be updated:

1.1 Use 2048-Bit Private Keys. For most web sites, security provided by 2,048-bit RSA keys is sufficient. The RSA public key algorithm is widely supported, which makes keys of this type a safe default choice. At 2,048 bits, such keys provide about 112 bits of security. If you want more security than this, note that RSA keys don't scale very well. To get 128 bits of security, you need 3,072-bit RSA keys, which are noticeably slower. ECDSA keys provide an alternative that offers better security and better performance. At 256 bits, ECDSA keys provide 128 bits of security. A small number of older clients don't support ECDSA, but modern clients do. It's possible to get the best of both worlds and deploy with RSA and ECDSA keys simultaneously if you don't mind the overhead of managing such a setup.

Answer 2

The key size for (symmetric) data encryption is negotiated as part of the ciphersuite, for example TLS_RSA_WITH_AES_256_CBC_SHA uses AES with 256-bit key while TLS_RSA_WITH_AES_128_CBC_SHA uses AES with 128-bit key. Java packages from Oracle (and previously Sun) out-of-the-box are limited to 128-bit strength symmetric crypto; to use AES-256 ciphersuites you need to install the 'Unlimited Strength' policy, see SSLHandshakeException while connecting to a https site but OpenJDK does not have this limitation. Note that strength is not the same as size; 3DES as used in SSL/TLS (keying option 1) has key size 168 but strength only 112 and is permitted under the 'limited' policy.

There are no '512 bit encryption' algorithms used in any SSL/TLS at all. Note that breaking 256-bit strength like AES-256 by conventional means cannot be done in our current solar system; you'd need to control at least much of the galaxy for billions of years, and to such an adversary your cat videos, bank accounts, or even business strategy are completely uninteresting. Quantum may change this, but no one knows if that will ever work, when or how well.

Yes, the RSA key is in the certificate, if you use RSA which is the most common choice but not the only one. (Except there were ephemeral RSA keys in some of the deliberately broken 'export' ciphersuites which you don't ever want to use and that have been officially prohibited in TLSv1.1 2006 and TLSv1.2 2008, and Java has disabled entirely by default since Java7 circa 2011.) Public CAs like Symantec GoDaddy etc. have not issued certificates for RSA keys smaller than 2048 bits since 2013; if you use selfsigned certs or run your own personal CA you can do so, but browsers will probably give errors or warnings.

Note that RSA-1024 has only about 80 bits of strength (which is why it is no longer considered secure) and RSA-2048 about 112; see https://www.keylength.com and
https://crypto.stackexchange.com/questions/1978/how-big-an-rsa-key-is-considered-secure-today
https://crypto.stackexchange.com/questions/6236/why-does-the-recommended-key-size-between-symmetric-and-assymetric-encryption-
https://crypto.stackexchange.com/questions/8687/security-strength-of-rsa-in-relation-with-the-modulus-size