PDF Signature Validation

Question

I am trying to validate a PDF Signature using PDFBox and BouncyCastle. My code works for most PDF:s, however there is one file, the cryptographic validation using BouncyCastle fails. I'm using pdfbox 1.8, BouncyCastle 1.52. The test input pdf file is randomly got from somewhere, it seems that it is generated using iText. Test pdf file

public void testValidateSignature() throws Exception
{
    byte[] pdfByte;
    PDDocument pdfDoc = null;
    SignerInformationVerifier verifier = null;
    try
    {
        pdfByte = IOUtils.toByteArray( this.getClass().getResourceAsStream( "SignatureVlidationTest.pdf" ) );
        pdfDoc = PDDocument.load( new ByteArrayInputStream( pdfByte ));
        PDSignature signature = pdfDoc.getSignatureDictionaries().get( 0 );

        byte[] signatureAsBytes = signature.getContents( pdfByte );
        byte[] signedContentAsBytes = signature.getSignedContent( pdfByte );
        CMSSignedData cms = new CMSSignedData( new CMSProcessableByteArray( signedContentAsBytes ), signatureAsBytes);
        SignerInformation signerInfo = (SignerInformation)cms.getSignerInfos().getSigners().iterator().next();
        X509CertificateHolder cert = (X509CertificateHolder)cms.getCertificates().getMatches( signerInfo.getSID() ).iterator().next();
        verifier = new JcaSimpleSignerInfoVerifierBuilder( ).setProvider( new BouncyCastleProvider() ).build( cert );

        // result if false
        boolean verifyRt = signerInfo.verify( verifier );

    }
    finally
    {
        if( pdfDoc != null )
        {
            pdfDoc.close();
        }
    }

}

Answer 1

In a comment to my other answer the OP asked a related question

This time it is adbe.pkcs7.detached signature, it also fails in cryptographic validation. I extracted the signedContent and signature, run a unit test with original BC source code, the failure is in Arrays.constantTimeAreEqual(sig, expected) when compare the signature with computed expected digest. test pdf

(Strictly speaking such a separate (if related) question should have been asked as a separate stack overflow question but it was interesting enough to investigate it nonetheless.)

TL; DR

The signed attributes in question are not properly DER encoded, they are merely presented in a different, also possible BER encoding. Some validators take the signed attributes as they are, some enforce DER encoding before validation. The latter ones, therefore, indirectly reject improperly encoded signed attributes. Adobe Reader is a sample of the former, BouncyCastle of the latter.

In detail

The improved method validateSignaturesImproved from my other answer also shows a validation failure but as a help it outputs the encoded signed attributes. This output, compared to the corresponding section of the signature container, shows the problem.

---

A bit of background:

All but the most primitive CMS signature containers don't sign the document data directly but instead a set of so called signed attributes which in turn include a hash value of the document data.

There are certain encoding rules for the data in signature containers embedded in PDF files. On one hand there are the basic encoding rules (BER) which allow different ways to encode the same type of data; e.g. the elements of sets may appear in any order. And there are the distinguished encoding rules (DER) which only allow a single way to encode given data; e.g. there is a pre-defined order in which elements of a set must be given.

According to the CMS specification RFC 5652:

SignedAttributes MUST be DER encoded, even if the rest of the structure is BER encoded.

(Section 5.3 - SignerInfo Type)

Strictly speaking the PDF specification ISO 32000-1 even is more strict, it requires:

When PKCS#7 signatures are used, the value of Contents shall be a DER-encoded PKCS#7 binary data object containing the signature. The PKCS#7 object shall conform to RFC3852 Cryptographic Message Syntax.

(Section 12.8.3.3 - PKCS#7 Signatures as used in ISO 32000)

(RFC 3852 is the precursor of RFC 5652 and obsoleted by it.)

Thus, for interoperable CMS signatures in PDFs the whole signature container has to be DER encoded.

---

The signature at hand indeed is not too trivial and uses signed attributes. The signature container contains these signed attributes:

SEQUENCE {
  OBJECT IDENTIFIER contentType (1 2 840 113549 1 9 3)
  SET {
    OBJECT IDENTIFIER data (1 2 840 113549 1 7 1)
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER signingTime (1 2 840 113549 1 9 5)
  SET {
    UTCTime 07/12/2016 16:11:08 GMT
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER
    signingCertificateV2 (1 2 840 113549 1 9 16 2 47)
  SET {
    SEQUENCE {
      SEQUENCE {
        SEQUENCE {
          OCTET STRING
            43 D1 C4 40 09 EB 32 46 B0 5C 2D A8 81 71 54 48
            F4 A3 9D 6F E3 6B 5C 9E 8F 4B 07 6D 10 55 D2 C8
          }
        }
      }
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER messageDigest (1 2 840 113549 1 9 4)
  SET {
    OCTET STRING
      E9 23 CC 92 ED 09 3B CE 51 78 DE 86 E0 F0 C8 6E
      9B CD 82 CB 35 A0 BC 66 38 BC 13 DE F3 7D C7 BC
    }
  }

but the correct DER order of these set elements would have been this:

SEQUENCE {
  OBJECT IDENTIFIER contentType (1 2 840 113549 1 9 3)
  SET {
    OBJECT IDENTIFIER data (1 2 840 113549 1 7 1)
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER signingTime (1 2 840 113549 1 9 5)
  SET {
    UTCTime 07/12/2016 16:11:08 GMT
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER messageDigest (1 2 840 113549 1 9 4)
  SET {
    OCTET STRING
      E9 23 CC 92 ED 09 3B CE 51 78 DE 86 E0 F0 C8 6E
      9B CD 82 CB 35 A0 BC 66 38 BC 13 DE F3 7D C7 BC
    }
  }
SEQUENCE {
  OBJECT IDENTIFIER signingCertificateV2 (1 2 840 113549 1 9 16 2 47)
  SET {
    SEQUENCE {
      SEQUENCE {
        SEQUENCE {
          OCTET STRING
            43 D1 C4 40 09 EB 32 46 B0 5C 2D A8 81 71 54 48
            F4 A3 9D 6F E3 6B 5C 9E 8F 4B 07 6D 10 55 D2 C8
          }
        }
      }
    }
  }

As you can see, the last two attributes are not in the correct DER order in the signature container.

---

When validating signature data, BouncyCastle first parses the signature container into an object representation and forgets the original bytes. To retrieve the signed attributes for hashing, it creates the DER encoding corresponding to the internal object representation. Thus, BouncyCastle hashes the latter set.

Adobe Reader, on the other hand, appears to take the signed attributes as they are encoded in the embedded signature container. Thus, it hashes the former set.

Apparently the original signing software also signed the signed attributes in the first (invalid!) order. Thus, Adobe Reader succeeds in verifying the signature while BouncyCastle doesn't.

Strictly speaking this is a bug in Adobe Reader. On the other hand there are many PDF signing products used in the real world which are too dumb to properly sort the signed attributes, so (also) accepting the signed attributes in the given order might just be the right approach, even though a warning about structural issues would be appropriate.

---

And it really is quite a pity that even a large signing service like DocuSign has yet to learn the basics of signature container creation.

Answer 2

Your code completely ignores the SubFilter of the signature. It is appropriate for signatures with SubFilter values adbe.pkcs7.detached and ETSI.CAdES.detached but will fail for signatures with SubFilter values adbe.pkcs7.sha1 and adbe.x509.rsa.sha1.

The example document you provided has been signed with a signatures with SubFilter value adbe.pkcs7.sha1.

For details on how signatures with those SubFilter values are created and, therefore, have to be validated, confer the PDF specification ISO 32000-1 section 12.8 Digital Signatures.

---

This is a slightly improved validation method:

boolean validateSignaturesImproved(byte[] pdfByte, String signatureFileName) throws IOException, CMSException, OperatorCreationException, GeneralSecurityException
{
    boolean result = true;
    try (PDDocument pdfDoc = PDDocument.load(pdfByte))
    {
        List<PDSignature> signatures = pdfDoc.getSignatureDictionaries();
        int index = 0;
        for (PDSignature signature : signatures)
        {
            String subFilter = signature.getSubFilter();
            byte[] signatureAsBytes = signature.getContents(pdfByte);
            byte[] signedContentAsBytes = signature.getSignedContent(pdfByte);
            System.out.printf("\nSignature # %s (%s)\n", ++index, subFilter);

            if (signatureFileName != null)
            {
                String fileName = String.format(signatureFileName, index);
                Files.write(new File(RESULT_FOLDER, fileName).toPath(), signatureAsBytes);
                System.out.printf("    Stored as '%s'.\n", fileName);
            }

            final CMSSignedData cms;
            if ("adbe.pkcs7.detached".equals(subFilter) || "ETSI.CAdES.detached".equals(subFilter))
            {
                cms = new CMSSignedData(new CMSProcessableByteArray(signedContentAsBytes), signatureAsBytes);
            }
            else if ("adbe.pkcs7.sha1".equals(subFilter))
            {
                cms = new CMSSignedData(new ByteArrayInputStream(signatureAsBytes));
            }
            else if ("adbe.x509.rsa.sha1".equals(subFilter) || "ETSI.RFC3161".equals(subFilter))
            {
                result = false;
                System.out.printf("!!! SubFilter %s not yet supported.\n", subFilter);
                continue;
            }
            else if (subFilter != null)
            {
                result = false;
                System.out.printf("!!! Unknown SubFilter %s.\n", subFilter);
                continue;
            }
            else
            {
                result = false;
                System.out.println("!!! Missing SubFilter.");
                continue;
            }

            SignerInformation signerInfo = (SignerInformation) cms.getSignerInfos().getSigners().iterator().next();
            X509CertificateHolder cert = (X509CertificateHolder) cms.getCertificates().getMatches(signerInfo.getSID())
                    .iterator().next();
            SignerInformationVerifier verifier = new JcaSimpleSignerInfoVerifierBuilder().setProvider(new BouncyCastleProvider()).build(cert);

            boolean verifyResult = signerInfo.verify(verifier);
            if (verifyResult)
                System.out.println("    Signature verification successful.");
            else
            {
                result = false;
                System.out.println("!!! Signature verification failed!");

                if (signatureFileName != null)
                {
                    String fileName = String.format(signatureFileName + "-sigAttr.der", index);
                    Files.write(new File(RESULT_FOLDER, fileName).toPath(), signerInfo.getEncodedSignedAttributes());
                    System.out.printf("    Encoded signed attributes stored as '%s'.\n", fileName);
                }

            }

            if ("adbe.pkcs7.sha1".equals(subFilter))
            {
                MessageDigest md = MessageDigest.getInstance("SHA1");
                byte[] calculatedDigest = md.digest(signedContentAsBytes);
                byte[] signedDigest = (byte[]) cms.getSignedContent().getContent();
                boolean digestsMatch = Arrays.equals(calculatedDigest, signedDigest);
                if (digestsMatch)
                    System.out.println("    Document SHA1 digest matches.");
                else
                {
                    result = false;
                    System.out.println("!!! Document SHA1 digest does not match!");
                }
            }
        }
    }
    return result;
}

(Excerpt from ValidateSignature.java)

This method considers the SubFilter value and properly handels signatures with SubFilter value adbe.pkcs7.sha1. It does not yet support adbe.x509.rsa.sha1 or ETSI.RFC3161 signatures / time stamps yet but at least gives appropriate output.