JavaMail with Apache James mail server

Question

I have an apache james mail server hosted on a local machine. It uses a self signed certificate which I added to the trusted list. I am trying to send and receive mails using java mail

I get this error:

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

The SSL debug output however says that a trusted certificate is found. Otherwise it doesn't tell me much. I should also mention that this mail server works with openssl as well as with thunderbird.

Code for the receiving of mails:

    String host = "192.168.1.21";
    Boolean debug = true;

    POP3Folder folder = null;
    Store store = null;
    try {
        Properties props = new Properties();
        props.put("mail.host", host);
        props.put("mail.store.protocol", "pop3s");
        props.put("mail.pop3s.port", "995");

      Session session = Session.getInstance(props,null);
      session.setDebug(debug);

      store = session.getStore("pop3s");
      store.connect(username, password);

The exception is thrown when I attempt to connect.

I have been banging my head against the wall for the last couple of hours/days so any help would be greatly appreciated.

EDIT:

The SSL debug output:

Info:   ***
Info:   Found trusted certificate:
Info:   [
[
Version: V3
Subject: CN=192.168.1.21, OU=private, O=private, L=pretoria, ST=gauteng, C=za
Signature Algorithm: SHA256withRSA, OID = 1.2.840.113549.1.1.11
Key:  Sun RSA public key, 2048 bits
modulus:  22201738425808301357843951429131863923295077691776461029270738957881925042102429206972015246280434827640419315658812269457485815395646018000726167885520466978079051879949885421741485411500412697981582621030362804785391242469536810788864680524659094190388912471585546967116467038492937424356023436763640787748242238829212068970215212531761712168559272937198654805596431568611192706600640030995533703350490664304506975658770991265086884832523665903150599863152070395170101007238711948275224105410201713594276436919539183706721126654808927498591115057177598201458589477257783098334024997797269658976390073190289972335957
public exponent: 65537
Validity: [From: Thu May 01 13:28:37 CAT 2014,
           To: Wed Jul 30 13:28:37 CAT 2014]
Issuer: CN=192.168.1.21, OU=private, O=private, L=pretoria, ST=gauteng, C=za
SerialNumber: [    618a1f7d]
Certificate Extensions: 1
[1]: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 90 DF D4 14 E8 B7 70 38   28 F0 7F CC 83 60 3E 98  ......p8(....`>.
0010: DC EB 0B D5                                        ....
]
]
]
Algorithm: [SHA256withRSA]
Signature:
0000: 13 42 F1 F0 FB C4 A4 AD   1B 93 96 CE 53 64 72 4A  .B..........SdrJ
0010: D2 C5 C7 66 18 BA 07 A6   C3 C6 97 9F E4 D1 8B 6F  ...f...........o
0020: B9 72 3C F6 1C 3F 98 FB   3C 6C 74 A3 20 83 99 9A  .r<..?..<lt. ...
0030: 9D 91 41 32 59 71 63 4A   3B 84 2E 2D 72 9F 2D AA  ..A2YqcJ;..-r.-.
0040: 83 84 56 78 19 F9 8A AF   DD 11 D5 C5 21 9E 93 06  ..Vx........!...
0050: 4D 48 2D 22 12 1F DA 1F   40 6A AD 9A 9A 29 4F 52  MH-"....@j...)OR
0060: 2D EB EB A7 13 B9 27 11   35 94 02 25 4E DF E5 6C  -.....'.5..%N..l
0070: 6B 12 79 DD 22 E9 BB FE   20 34 4F B4 A1 CE E2 14  k.y."... 4O.....
0080: EE A4 B4 A8 D5 2D 9F 80   82 5E 71 03 49 B3 30 3C  .....-...^q.I.0<
0090: 56 06 E3 62 2E 1C 5A E4   EE 15 4A 03 77 1C 94 4C  V..b..Z...J.w..L
00A0: 20 D7 47 95 62 7F 21 22   CB 64 BF A0 34 D6 D5 AD   .G.b.!".d..4...
00B0: 57 C1 A3 AD 69 70 DB 32   A5 B6 38 BB 1F 00 C7 5A  W...ip.2..8....Z
00C0: 3A 73 3B 8D EE 2E A8 40   9A 24 D0 58 5C D5 A4 2D  :s;....@.$.X\..-
00D0: 0F 09 2E DB 84 CF 55 21   79 C8 22 B5 2D E7 91 51  ......U!y.".-..Q
00E0: 05 8A 7D 1A 19 25 CC 30   EC 9B BA 77 78 9E 2E C9  .....%.0...wx...
00F0: 6C 2D F3 47 E9 44 1E 5A   41 92 14 11 9B E4 8E 59  l-.G.D.ZA......Y
]
Info:   *** ServerHelloDone
Info:   *** ClientKeyExchange, RSA PreMasterSecret, TLSv1
Info:   http-listener-1(2), WRITE: TLSv1 Handshake, length = 262
Info:   SESSION KEYGEN:
Info:   PreMaster Secret:
Info:   0000:
Info:   03 
//infos continue with things in between like  CONNECTION KEYGEN: etc
//many more things like this
//continued
http-listener-1(2), WRITE: TLSv1 Change Cipher Spec, length = 1
Info:   *** Finished
Info:   verify_data:  {
Info:   121
Info:   ,
Info:   89
//many more infos

Info:   }
Info:   ***
Info:   http-listener-1(2), WRITE: TLSv1 Handshake, length = 48
Info:   http-listener-1(2), READ: TLSv1 Alert, length = 2
Info:   http-listener-1(2)
Info:   , RECV TLSv1 ALERT:
Info:   fatal,
Info:   handshake_failure
Info:   %% Invalidated:  [Session-2, TLS_RSA_WITH_AES_128_CBC_SHA]
Info:   http-listener-1(2), called closeSocket()
Info:   http-listener-1(2), handling exception: javax.net.ssl.SSLHandshakeException: Received fatal   alert: handshake_failure

Answer 1

How exactly did you add the certificate to the "trusted list"?

Have you seen this JavaMail FAQ entry?

You can also try setting the mail.pop3s.ssl.trust property to "*" or to the name of your server.

BTW, in your code, you don't need to set mail.store.protocol or mail.pop3s.port. The former is not needed because you pass the protocol name to the getStore method explicitly. The latter is not needed because it's the default for the "pop3s" protocol.

I checked with a JSK SSL expert, who had this to say:

There is so much missing from this log, it's hard to tell what's really gone on.

There seems to be a problem with the decrypt/de-pad/de-MAC operation on the server side.

They took out the most important byte (second) of the RSA Premaster Secret:

Info:   PreMaster Secret:
Info:   0000:
Info:   03

If I had to guess, I would suggest trying:

java -Dcom.sun.net.ssl.rsaPreMasterSecretFix=true App

then switch to false.

Other comments:

EJP seems to think this is a rehandshake, the only hint is "Session-2". There could have been a second handshake on this connection, but those are usually only done in the case of asking for Client Authentication, but there is no CertificateRequest between the Certificate and the ServerHelloDone, so probably not that.

This could very likely be just the second separate connection made by this process.

This has nothing to do with trust, the handshake wouldn't proceed past the ServerHelloDone if that didn't succeed.

In the case of any handshake, the client sends the ChangeCipherSpec, then the next packet is a Finished packet with verify_data which is encrypted using the just negotiated keys (48 bytes = 4 Header + 12 verify_data + 20 MAC + 12 padding). If the server can't decrypt/de-pad(AES-CBC)/de-MAC properly, then it will send back a handshake_failure, which seems to be the case.

There seems to be a problem with that decrypt/de-pad/de-MAC operation. The problem could be on the server side (most likely), or perhaps they put in a replacement provider on the client side?