CODEWITHSUNDEEP
azure-cognitive-servicesazure-form-recognizer
rdodhia
rdodhia

Reputation: 350

Recognizing checkboxes in azure form recognizer

The forms I want to use in form recognizer have several checkboxes. Currently, form recognizer's OCR engine doesn't seem to detect checkboxes. Is there a work-around for this, or is there a way to train checkbox recognition?

I've also tried using tesseract on windows and linux, but I couldn't solve it there either.

The checkboxes are your standard open squares. Sometimes they may have an "x" or checkmark on them.

If there is no solution, then I may ask the customer to change from checkboxes to filled in circles (radio-buttons). I haven't tested that yet, to see what OCR would make of them.

Edit: I read the form recognizer documentation and I saw that it explicitly says checkboxes and radio buttons are not supported. I wonder if anyone has a awork-around?

Upvotes: 1

Views: 3537

Answers (4)

Adam D
Adam D

Reputation: 161

Support for checkboxes was added to Form Recognizer in version 2.1 (in public preview as of September 2020). From the announcement:

Checkbox / Selection Mark detection – Form Recognizer supports detection and extraction of selection marks such as check boxes and radio buttons. Selection Marks are extracted in Layout and you can now also label and train in Train Custom Model - Train with Labels to extract key value pairs for selection marks.

There is now a selectionMarks object in the Get Analyze Layout Result API response that lists detected selection marks and their state, either selected or unselected.

Support for labeling checkboxes and selection marks was also added to the sample labeling tool as of version 2.1.

Upvotes: 2

Neil Stenton
Neil Stenton

Reputation: 143

The method @Ram-msft suggested can work, but I find that the recognizer struggles to consistently pick out any single characters in a box - although to be honest checkbox type boxes seem to work better than say a number in a box (in my experience at least).

As long as you're interested in any "value" inside the box (i.e. it's not empty) then that method should give you reasonable results until they come up with a true solution.

Upvotes: 1

Ram
Ram

Reputation: 2754

In Form Recognizer if the forms have a consistent layout, you might be able to tag the area using the new GUI tool and pass that specific area to OCR to try to improve results.

Upvotes: 1

Nicolas R
Nicolas R

Reputation: 14619

An OCR is by definition made for "character recognition" (see here). A checkbox or a radio button is not a character, so it will not be recognized by an OCR.

You can still give a try to a custom vision detector to find those items, but it will be a bit complicated as this service will only find those items in your document and you will have to combine with an OCR call to get the text, then try to match the zones in the document to know which text is associated with which combo/radio button

Upvotes: 3

Related Questions