CODEWITHSUNDEEP
node.jsazure-language-understandingbotframework
jano
jano

Reputation: 765

Botframework responses with intent with lower score

I have a bot which uses two LUIS apps as a LuisRecognizers to guess the client intent. My question is why does the bot respond the intent which has the lowest score? I double checked this and if i manually check the score through the Luis dashbord then i received something like: IntentA with score 0.92 and IntentB with score 1. And if i pass the same input through the botframework it responses with IntentA which has lower score. Am i missing something? I tried to play with intentThreshold, recognizeMode or recognizeOrder, all mentioned in docs, however not received better results.

Upvotes: 0

Views: 893

Answers (2)

jbeltran
jbeltran

Reputation: 214

Have you tried from the LUIS dashboard your published model? I had the same problem because LUIS wasn't publishing my model correctly at the moment and it didn't catch the changes I made, so the trained model worked perfectly in the dashboard but the published not.

I tried the next day and it published everything correctly in both, the dashboard and the botframework.

Upvotes: 0

OmG
OmG

Reputation: 18858

If you consider the C# code of BotFramework you can see "the best intent from" function was implemented like the following:

protected virtual IntentRecommendation BestIntentFrom(LuisResult result)
{
    return result.Intents.MaxBy(i => i.Score ?? 0d);
}

If you want to test this, you can override it in your LuisDialog, to see the details of its mechanism (by logging scores of intetnts). as you can see the max score will be chosen at the decision point. Also, you can find the Luis recognizer in NodeJs:

LuisRecognizer.recognize(utterance, model, (err, intents, entities) => {
                if (!err) {
                    result.intents = intents;
                    result.entities = entities;

                    // Return top intent
                    var top: IIntent;
                    intents.forEach((intent) => {
                        if (top) {
                            if (intent.score > top.score) {
                                top = intent;
                            }
                        } else {
                            top = intent;
                        }
                    });
                    if (top) {
                        result.score = top.score;
                        result.intent = top.intent;

                        // Correct score for 'none' intent
                        // - The 'none' intent often has a score of 1.0 which
                        //   causes issues when trying to recognize over multiple
                        //   model. Setting to 0.1 lets the intent still be 
                        //   triggered but keeps it from trompling other models.
                        switch (top.intent.toLowerCase()) {
                            case 'builtin.intent.none':
                            case 'none':
                                result.score = 0.1;
                                break;
                        }
                    }
                    cb(null, result);
                } else {
                    cb(err, null);
                }
            });

Again the same as C# code, the recognizer choosees the max score, if there exists an application model in Luis. Therefore, this problem does not come from the client. Hence, a suggestion can be considering the JSON response of the LUIS which is received to your client.

Upvotes: 1

Related Questions