CODEWITHSUNDEEP
javaitextpdfbox
user6404269
user6404269

Reputation: 33

Parse PDF table and display it as CSV(Java)

I am trying to parse a TABLE in PDF file and display it as CSV. I have attached sample data from PDF below(only few columns) and sample output for the same. Each column width is fixed, let's say Company Name(18 char),Amount(8 char), Type(5 char) etc. I tried using Itext and PDFBox jars to get each page data and parsed line by line, but sounds like it is not a great solution as the line breaks and page breaks in PDF are not proper. Please me let me know if there is any other appropriate solution. We want to use any open source software for this.

Upvotes: 1

Views: 1563

Answers (1)

Joris Schellekens
Joris Schellekens

Reputation: 9057

This is a very complex problem. There are multiple master dissertations about this even.

An easy analogy: I have 5000 puzzle-pieces, all of them are perfectly square and could fit anywhere. Some of them have pieces of lines on them, some of them have snippets of text.

However, that does not mean it can't be done. It'll just take work.

General approach:

  1. use iText (specifically IEventListener) to get information on all rendering events for every page
  2. select those rendering events that make sense for your application. PathRenderInfo and TextRenderInfo.
  3. Events in a pdf do not need to appear in order according to the spec. Solve this problem by implementing a comparator over IEventData. This comparator should sort according to reading order. This implies you might have to implement some basic language detection, since not every language reads left-to-right.
  4. Once sorted, you can now start clustering items together according to any of the various heuristics you find in literature. For instance, two characters can be grouped into a snippet of text if they follow each other in the sorted list of events (meaning they appear next to each other in reading order), if the y-position does not differ too much (subscript and superscript might screw with this), and if the x-position does not differ too much (kerning).
  5. Continue clustering characters until you have formed words
  6. Assuming you have formed words, use similar algorithm to form words into lines. Use PathRenderInfo to withhold merging words if they intersect with a line.
  7. Assuming you have managed to create lines, now look for tables. One possible approach is apply a horizontal and vertical projection. And look for those sub-areas in the page that (when projected) show a grid-like structure.

This high-level approach should make it painfully obvious why this is not a widely available thing. It's very hard to implement. It requires domain-knowledge of both PDF, fonts, and machine-learning.

If you are ok with commercial solutions, try out pdf2Data. It's an iText add-on that features this exact functionality.

http://itextpdf.com/itext7/pdf2Data

Upvotes: 3

Related Questions