CODEWITHSUNDEEP
botframeworkazure-qna-maker
Druudik
Druudik

Reputation: 1015

Is there any way to specify order of alternate questions in QnA pair (QnA Maker)?

I am using QnA Maker Service and also active learning feature via explicit feedback which works as follows: query QnA Maker, find ambigous questions/answer pairs, create hero card by choosing first question from each QnA pair (QnAMakerResult) and send it to the user.

Since I am using this logic I want to have the best looking first question in QnA pair. But I have found out that when I click on Save and Train in QnA Maker portal, it will override my order of alternate questions in qna pair which completely breaks active learning experience because some questions in qna pair are not so user friendly such as only specific keywords etc.

Is there any way how to mark some alternate question for this purpose (like mark the most user friendly question) or to disable this automatic sorting in QnA Maker?

Upvotes: 2

Views: 276

Answers (1)

Matt Stannett
Matt Stannett

Reputation: 2728

Since you haven't specified a language, my answer refers to the C# SDK, but the concepts should be transferable to the other SDKs.

I believe you might be able to achieve the functionality that you want in a roundabout and somewhat manual way:

  1. In the QnA Maker Portal add a metadata property to the questions you'd like to disambiguate, it could have the key disambiguation-question and the value would be the question you'd like to use for disambiguation. An example is provided here. e.g. disambiguation-question: My question that I want to use to disambiguate.
  2. Save, Train, and Publish your Knowledge base.
  3. Update your code to filter on the Metadataproperty of the QueryResult class, which contains an array of Metadata objects. TheGetAnswersAsyncmethod which is how you query your QnA Maker endpoint, will return an array ofQueryResult` objects.
  4. Use the Value from the Metadata object to display in your card for disambiguation. There is an example of how to do this here.

There is also an overview of how the QnA Maker ranking system for answers works available here, currently it works as follows:

| Step  | Purpose |
| --- | --- |
| 1 | The client application sends the user query to the GenerateAnswer API |
| 2 | QnA Maker preprocesses the user query with language detection, spellers, and word breakers. |
| 3 | This preprocessing is taken to alter the user query for the best search results. |
| 4 | This altered query is sent to an Azure Cognitive Search Index, which receives the top number of results. If the correct answer isn't in these results, increase the value of top slightly. Generally, a value of 10 for top works in 90% of queries. |
| 5 | QnA Maker uses syntactic and semantic based featurization to determine the similarity between the user query and the fetched QnA results. |
| 6 | The machine-learned ranker model uses the different features, from step 5, to determine the confidence scores and the new ranking order. |
| 7 | The new results are returned to the client application in ranked order. |

Features used include but aren't limited to word-level semantics, term-level importance in a corpus, and deep learned semantic models to determine similarity and relevance between two text strings.

I hope this helps.

Upvotes: 2

Related Questions