Parsing PDF files

Question

I'm finding it difficult to parse a pdf file that's created in a non-english language. I used pdfbox and itext but couldn't find anything in there that could help parse this file. Here's the pdf file that I'm talking about: http://prapatti.com/slokas/telugu/vishnusahasranaamam.pdf The pdf says that it's created use LaTeX and Tikkana font. I have Tikkana font installed on my machine, but that didn't help. Please help me in this.

Thanks, K

Answer 1

When you say "parse PDF files", my first thought was that the PDF in question wasn't opening in various PDF viewers & libraries, and was therefore corrupt in some way.

But that's not the case at all. It opens just fine in Acrobat Reader X. And then I see the text on the page.

And when I copy/paste that text from the first page, I get:

Ûûp{¨¶ðQ{p{¨|={pÛû{¨>üb¶úN}l{¨d{p{¨> >Ûpû¶bp{¨}|=/}pT¶=}Nm{Z{Úpd{m}a¾Ú}mp{Ú¶¨>ztNð{øÔ_c}m{ТÁ}=N{Nzt¶ztbm}¥Ázv¬b¢Á Á ÛûÁøÛûzÏrze¨=ztTzv}lÛzt{¨d¨c}p{Ðu{¨½ÐuÛ½{=Û Á{=Á Á ÁÛûb}ßb{q{d}p{¨ze=Vm{Ðu½Û{=Á

That's from Reader.

Much of the text in this PDF is written using various "Type 3" fonts. These fonts claim to use "WinAnsiEncoding" (Also Known As code page 1252), with a "differences" array. This differences array is wrong:

47 /BB 61 /BP /BQ 81 /C6...

The first number is the code point being replaced, the second is a Name of a character that replaces the original value at that code point.

There's no such character names as BB, BP, BQ, C9... and so on. So when you copy-paste that text, you get the above garbage.

I'm sorry, but the only reliable way to extract text from such a PDF is OCR (optical character recognition).

Eh... Long shot idea:

If you can find the specific versions of the specific fonts used to generate this PDF, you just might be able to determine the actual stream contents of known characters converted to Type 3 fonts in this way.

Once you have these known streams, you can compare them to the streams in the PDF and use that to build your own translation table.

You could either fix the existing PDF[s] (by changing the names in the encoding dictionary and Type 3 charproc entries) such that these text extractors will work correctly, or just grab the bytes out of the stream and translate them yourself.

The workflow would go something like this:

1. For each character in a font used in the form:
   1. render it to PDF by itself using the same LaTeK/GhostScript versions.
   2. Open the PDF and find the CharProc for that particular known character.
   3. Store that stream along with the known character used to build it.
2. For each text byte in the PDF to be interpreted.
   1. Get the glyph name for the given byte based on the existing encoding array
   2. Get the "char proc" stream for that glyph name and compare it to your known char procs.

NOTE: This could be rewritten to be much more efficient with some caching, but it gets the idea across (I hope).

All that requires a fairly deep understanding of PDF and the parsing methods involved. But it just might work. Might not too...