pdfbox embedding subset font for annotations

Question

I am trying to use Apache PDFBOX v2.0.21 to modify existing PDF documents, adding signatures and annotations. That means that I am actively using incremental save mode. I am also embedding LiberationSans font to accommodate some Unicode characters. It makes sense for me to use the subsetting feature of PDF embedded fonts as embedding LiberationSans in full makes the PDF file around 200+ KB more in side.

After multiple trials and errors I finally managed to have something working - all but the font subsetting. The way I do this is to initialize the PDFont object once using

  try (InputStream fs = PDFService.class.getResourceAsStream("/static/fonts/LiberationSans-Regular.ttf")) {
     _font = PDType0Font.load(pddoc, fs, true);
  }

And then to use custom Appearance Stream to show the text.

   private void addAnnotation(String name, PDDocument doc, PDPage page, float x, float y, String text) throws IOException {
      
      List<PDAnnotation> annotations = page.getAnnotations();

      PDAnnotationRubberStamp t = new PDAnnotationRubberStamp();

      t.setAnnotationName(name); // might play important role
      t.setPrinted(true); // always visible
      t.setReadOnly(true); // does not interact with user
      t.setContents(text); 
      
      PDRectangle rect = ....;
      t.setRectangle(rect);

      PDAppearanceDictionary ap = new PDAppearanceDictionary();
      ap.setNormalAppearance(createAppearanceStream(doc, t));
      ap.getCOSObject().setNeedToBeUpdated(true);
      t.setAppearance(ap);
      
      annotations.add(t);
      page.setAnnotations(annotations);
      
      t.getCOSObject().setNeedToBeUpdated(true);
      page.getResources().getCOSObject().setNeedToBeUpdated(true);
      page.getCOSObject().setNeedToBeUpdated(true);
      doc.getDocumentCatalog().getPages().getCOSObject().setNeedToBeUpdated(true);
      doc.getDocumentCatalog().getCOSObject().setNeedToBeUpdated(true);      
   }
   
   private PDAppearanceStream createAppearanceStream(final PDDocument document, PDAnnotation ann) throws IOException
   {
      PDAppearanceStream aps = new PDAppearanceStream(document);
      PDRectangle rect = ann.getRectangle();
      rect = new PDRectangle(0, 0, rect.getWidth(), rect.getHeight());
      aps.setBBox(rect); // set bounding box to the dimensions of the annotation itself
     
      // embed our unicode font (NB: yes, this needs to be done otherwise aps.getResources() == null which will cause NPE later during setFont)
      PDResources res = new PDResources();
      _fontName = res.add(_font).getName();
      aps.setResources(res);

      PDAppearanceContentStream apsContent = null;
      
      try {
         // draw directly on the XObject's content stream
         apsContent = new PDAppearanceContentStream(aps);

         apsContent.beginText();
         apsContent.setFont(_font, _fontSize);         
         apsContent.showText(ann.getContents());
         apsContent.endText();
      }
      finally {
         if (apsContent != null) {
            try { apsContent.close(); } catch (Exception ex) { log.error(ex.getMessage(), ex); }
         }
      }      

      aps.getResources().getCOSObject().setNeedToBeUpdated(true);
      aps.getCOSObject().setNeedToBeUpdated(true);
      return aps;
   }     

This code runs, but creates a PDF with dots instead of actual characters, which, I guess, means that the font subset has not been embedded. Moreover, I get the following warnings:

2021-04-17 12:33:31.326 WARN 20820 --- [ main] o.a.p.pdmodel.PDAbstractContentStream : attempting to use subset font LiberationSans without proper context

After looking through the source code, I get and I guess that I am messing something up when creating the appearance stream - somehow it's not connected with the PDDocument and the subsetting does not continue normally. Note that the above code works well when the font is embedded fully (i.e. if I call PDType0Font.load with the last parameter set to false)

Can anyone think of some hint to give to me? Thank you!

Answer 1

I don't know - am I lucky? It is very often that luckiness in programming points to something completely wrong or misleading. In any case, if someone can still give a hint, my ears are more than open...

Again, after looking through the code, I saw the following in PDDocument.save():

// subset designated fonts
for (PDFont font : fontsToSubset)
{
    font.subset();
}

This is not happening in PDDocument.saveIncremental() which I am using. Just to mess around with the code, I went and did the following just before calling saveIncremental() on my document:

 _font.subset(); // you can see in the beginning of the question how _font is created
 _font.getCOSObject().setNeedToBeUpdated(true);
 pddoc.saveIncremental(baos);

Believe it or not, but the document was saved correctly - at least it appears correct in Acrobat Reader DC and Chrome & Firefox PDF viewers. Note that Unicode codepoints are added to the subset for the font during showText() on appearance content stream.

UPDATE 18/04/2021: as I mentioned in the comments, I got reports from users that started seeing messages like "Cannot extract the embedded font XXXXXX+LiberationSans-Regular from ...", when they opened the modified PDF files. Strangely enough, I didn't see these messages during my tests. It turns out that my copy of Acrobat Reader DC was newer than theirs, and specifically with the continuous release version 2021.001.20149 no errors were shown, while with the continuous release version 2020.012.20043 the above message was shown.

After investigations, it turns out that the problem was with the way I was embedding the font. I am not aware if any other way exists, and I am not that familiar with the PDF specification to know otherwise. What I was doing, as you can see from the above code, was to load the font ONCE for the document, and then to use it freely in the resource dictionary of the appearance stream of EVERY annotation. This had as a result all the resource dictionaries of the annotation content streams to reference an F1 font that was defined with the SAME /BaseFont name. The PDF Reference, 3rd ed. on p.323 specifically states that:

"... the PostScript name of the font - ... - begins with a tag followed by a plus sign (+). The tag consists of exactly six uppercase letters; the choice of letters is arbitrary, but different subsets in the same PDF file must have different tags..."

Once I started to call PDType0Font.load for each of my annotations and calling subset() (and of course setNeedToBeUpdated) after creating appearance stream for each of them, I saw that the BaseName attributes started to look indeed differently - and indeed, the older 2020 version of Acrobat Reader DC stopped complaining.

[edit 07/10/2021: even trying to use a single PDFont object per page (having multiple annotations with this font), and subsetting it once, after having called showText on appearances of all annotations, appears to not work - it appears that the subsetting uses the letters I passed to the first showText, and not the others, resulting in wrong rendering of the 2nd, 3rd etc. annotations that might have characters that didn't exist in the 1st annotation - so I reiterate that what worked was to use loadFont for each separate annotation and then (after modifying appearance with showText, which will mark the letters to be used during subsetting) to call subset() on each of these fonts (which will result in the change of the font name)]

Note that other than using iText RUPS for inspecting the PDF contents, one could use Foxit PDF viewer to at least ensure that the subset font names are different. Acrobat Reader DC and PDF-xChange in Properties -> Fonts just show the initial font name, like LiberationSans, without showing the 6-letter unique prefix.

UPDATE 19/04/2021 I am still working on this issue - because I still get reports about the infamous "Cannot extract the embedded font" message. It is quite possible that the original cause of that message was not (or not only) the fact that the different subsets had same BaseFont names. One thing that I am observing is that on some computers, the stamp annotations that I am using cause Acrobat Reader DC to open automatically the so called "Comments pane" - there are options to turn this automatic thing off (Preferences -> Commenting -> Show comments pane when a PDF with comments is opened). When this pane opens, either manually or automatically, the error message appears (and I was on my wits ends to see why same version of Acrobat Reader DC behaves differently for different machines). I think that Acrobat Reader tries to extract the full version of the font and fails, since it is only a subset. But, I guess, this doesn't have to do with the semantic contents of the document - the document still passes "qpdf --check". I am currently trying to find if it is possible to restrict stamps to not allow comments - i.e. some way to disable the comments pane in Acrobat Reader DC, although I have little hope.

UPDATE 20/04/2021 opened a new question here