CODEWITHSUNDEEP
azure-language-understanding
deeepss
deeepss

Reputation: 153

Applying threshold to LUIS like Kore.ai ( Definitive, Possible and Eliminated)

I have a web application that calls LUIS and Kore.ai for Intent recognition. In Kore, I was able to set thresholds for definitive, possible and eliminated. Unfortunately,there is no such option in LUIS and I end up using the top scoring Intent. I defined a threshold myself so that I can match up like Kore responses.

But, few intents (recognition through LUIS) fall below the threshold (set like we do in Kore) and gets eliminated but LUIS gives top scoring Intent correctly. How do we handle this?

Is the logic of setting a threshold for LUIS results (like Kore) is correct ? Or Should I use LUIS's top scoring intent as Definitive.

Upvotes: 0

Views: 130

Answers (1)

Steven Kanberg
Steven Kanberg

Reputation: 6383

It's understandable that you want to try and equate the two services, but you shouldn't. The methodologies, algorithms, etc., used by each service appear to be too different (hence, LUIS returning most intents above the threshold you are trying to align with Kore.ai.

If you're plan is to use both LUIS and Kore.ai, then you should work with each as they were designed. In other words, use LUIS's top scoring intent.

Alternatively, you could build a set of thresholds for LUIS that produce similar results as Kore.ai, but don't align by value. For example, if definitive captures the top two then adjust the LUIS threshold to do the same.

I wouldn't actually recommend this as it would likely require a fair amount of setup and there is no guarantee the values will necessarily align with the thresholds created to match Kore.ai as LUIS results are non-deterministic.

Upvotes: 1

Related Questions