Document Intelligence custom model not applying

Question

I am new to coding and trying to use my custom extraction template model in python file to read more documents. I made the model using Document Intelligence Studio, applied my own labels, trained it and tested and it seemed to work fine. I tried using model ID to read pdfs from my local folder using python app and it didn't analyse documents with the labels from the model. Instead it seemed that it used auto-labeling which I don't want because I would've used a sample model then. Is the fault in my code? Or maybe something else? Here it is (I mainly used the code provided by Document Intelligence Studio, maybe I should change it?):

endpoint = "<myendpoint>"
key = "<mykey>"

model_id = "<my-custom-model-id>"
formUrl = "<pdf-url>"

document_analysis_client = DocumentAnalysisClient(
    endpoint=endpoint, credential=AzureKeyCredential(key)
)

# Initialize the DocumentAnalysisClient with endpoint and key
document_analysis_client = DocumentAnalysisClient(
        endpoint=endpoint, credential=AzureKeyCredential(key)
)

# Make sure your document's type is included in the list of document types the custom model can analyze
poller = document_analysis_client.begin_analyze_document_from_url(model_id, formUrl)
result = poller.result()

for idx, document in enumerate(result.documents):
    print("--------Analyzing document #{}--------".format(idx + 1))
    print("Document has type {}".format(document.doc_type))
    print("Document has confidence {}".format(document.confidence))
    print("Document was analyzed by model with ID {}".format(result.model_id))
    for name, field in document.fields.items():
        field_value = field.value if field.value else field.content
        print("......found field of type '{}' with value '{}' and with confidence {}".format(field.value_type, field_value, field.confidence))


# iterate over tables, lines, and selection marks on each page
for page in result.pages:
    print("\nLines found on page {}".format(page.page_number))
    for line in page.lines:
        print("...Line '{}'".format(line.content.encode('utf-8')))
    for word in page.words:
        print(
            "...Word '{}' has a confidence of {}".format(
                word.content.encode('utf-8'), word.confidence
            )
        )
    for selection_mark in page.selection_marks:
        print(
            "...Selection mark is '{}' and has a confidence of {}".format(
                selection_mark.state, selection_mark.confidence
            )
        )

for i, table in enumerate(result.tables):
    print("\nTable {} can be found on page:".format(i + 1))
    for region in table.bounding_regions:
        print("...{}".format(i + 1, region.page_number))
    for cell in table.cells:
        print(
            "...Cell[{}][{}] has content '{}'".format(
                cell.row_index, cell.column_index, cell.content.encode('utf-8')
            )
        )
print("-----------------------------------")

I tried changing types of models I have on Document Intelligence Studio but nothing helped, so I figured it has to do something with my code... maybe I have to specify labeling? But dunno...

Answer 1

Training a custom model with specific labeling, it should be able to handle the types of data it was trained on. Check the labeling in Document Intelligence Studio matches the data structure of the PDFs being analyzed.

Here I have trained a custom model in Document Intelligence Studio to recognize these key-value pairs.

• Added labels in the code to extract and print these pairs from the document analysis results.

Code:

import os
from azure.core.credentials import AzureKeyCredential
from azure.ai.formrecognizer import DocumentAnalysisClient

# Replace with your endpoint and key
endpoint = "https://doc inexxx.cognitiveservices.azure.com/"
key = "699a1e68fxxxxxxxxxxxx683a44d"
model_id = "testmodel2"

# Initialize the client
client = DocumentAnalysisClient(endpoint=endpoint, credential=AzureKeyCredential(key))

# Function to analyze a document using the custom model
def analyze_document(file_path):
    with open(file_path, "rb") as f:
        poller = client.begin_analyze_document(model_id=model_id, document=f)
        result = poller.result()

    print("Full Analysis Result:")
    print(result)
    
    for page in result.pages:
        print(f"Page number: {page.page_number}")
        
        if hasattr(page, 'key_value_pairs'):
            print("Key-Value Pairs:")
            for kv_pair in page.key_value_pairs:
                key = kv_pair.key.content if kv_pair.key else "None"
                value = kv_pair.value.content if kv_pair.value else "None"
                print(f"  Key: {key}, Value: {value}")

        if hasattr(page, 'lines'):
            print("Lines:")
            for line in page.lines:
                print(f"  {line.content}")

        if hasattr(page, 'words'):
            print("Words:")
            for word in page.words:
                print(f"  Word: {word.content}, Confidence: {word.confidence}")

        if hasattr(page, 'selection_marks'):
            print("Selection Marks:")
            for selection_mark in page.selection_marks:
                print(f"  Selection Mark: {selection_mark.state}, Confidence: {selection_mark.confidence}")

        if hasattr(page, 'tables'):
            print("Tables:")
            for table in page.tables:
                for cell in table.cells:
                    print(f"  Cell[{cell.row_index}][{cell.column_index}]: {cell.content}")

# Path to your single PDF file
pdf_path = r"C:\Users\v-chikkams\Downloads\Get_Started_With_Smallpdf.pdf"

# Analyze the document
print(f"Analyzing document: {pdf_path}")
analyze_document(pdf_path)

Custom trained model:

Output: