CODEWITHSUNDEEP
azureazure-functionsazure-cognitive-searchazure-cognitive-servicesazure-search-.net-sdk
nerdHunter
nerdHunter

Reputation: 35

how to map distinct custom skillset to index

trying to add custom skill in the skillset and map it in the index

here is in detail

I'm using the azure Named Entity Recognition in my skillset as

    {
        "@odata.type": "#Microsoft.Skills.Text.MergeSkill",
        "description": "Merge text content with image tags",
        "insertPreTag": " ",
        "context": "/document",
        "inputs": [
            {
                "name": "text",
                "source": "/document/fullTextAndCaptions"
            },
            {
                "name": "itemsToInsert",
                "source": "/document/normalized_images/*/Tags/*/name"
            }
        ],
        "outputs": [
            {
                "name": "mergedText",
                "targetName": "finalText"
            }
        ]
    }

and in the indexer as

    {
        "sourceFieldName": "/document/finalText/pages/*/entities/*/value",
        "targetFieldName": "entities"
    },
    {
        "sourceFieldName": "/document/finalText/pages/*/locations/*",
        "targetFieldName": "locations"
    },

and it works 100% now I want to add the Distinct custom skill from https://github.com/Azure-Samples/azure-search-power-skills/tree/master/Text/Distinct I did publish the function and when I go to test it manually it works as expected. however overall its not working in skillset. I want it to take the location and filter it and output the distinct only in it's own field in the search index. I'm having a really hard time to configure the skillset and indexer to get it to work.

any help please?

Upvotes: 0

Views: 590

Answers (2)

8163264128
8163264128

Reputation: 787

You'll need to add the distinct custom skill like this, assuming you want to dedup over the whole document

{
    "skills": [
        ...

        {
            "@odata.type": "#Microsoft.Skills.Custom.WebApiSkill",
            "description": "Distinct skill",
            "uri": "<https://distinct-skill>",
            "context": "/document",
            "inputs": [
                {
                    "name": "locations",
                    "source": /document/finalText/pages/*/locations/*"
                }
            ],
            "outputs": [
                {
                    "name": "distinct",
                    "targetName": "distinctLocations"
                }
            ]
        }

        ...
    ]
}

and an output field mapping to put it into the index.

    {
        "sourceFieldName": "/document/distinctLocations",
        "targetFieldName": "distinctLocations"
    }

See https://learn.microsoft.com/en-us/azure/search/cognitive-search-custom-skill-interface#consuming-custom-skills-from-skillset for adding a custom skill.

Upvotes: 1

Bertrand Le Roy
Bertrand Le Roy

Reputation: 17814

The skill inputs for the custom skill must be configured to point to the data you want to disambiguate. In this case, you didn't really need to modify the code, all you had to do was have an input with name 'words' and source '/document/finalText/pages//locations/'.

Upvotes: 0

Related Questions