CODEWITHSUNDEEP
azure-language-understanding
Anurag Jain
Anurag Jain

Reputation: 11

How can I ensure consistency between two environments of Luis like Production and Non production

  1. After importing a luis model from one environment to another (e.g. UAT to production) as a new version, retraining and publishing, why score for same phrase do not match in both environments?
  2. What are the other parameters which impact the luis model scores apart from LUIS model defined with UI or with importing another luis model (json import)?
  3. How can I ensure consistency in two environments?

Upvotes: 0

Views: 119

Answers (1)

Jyo Fanidam
Jyo Fanidam

Reputation: 1230

Scores are not absolute, and only have meaning relative to other scores in the same request. LUIS training is non-deterministic, so between versions, and even between exporting and re-importing the exact same version of the app, an application and its models will not necessarily return the exact same scores. Your system should use the highest scoring intent regardless of its value. For example, a score below 0.5 does not necessarily mean that LUIS has low confidence. Providing more training data can help increase the score of the most-likely intent.

And to answer your third question, you can have both environments in the same app instead of having different apps by importing. You can publish your app to the Staging slot(safe environment to test new thing you are adding to your LUIS app, without breaking the Production Slot) or the Production Slot(stable version of your app that will be used by your end users). By using two publishing slots, this allows you to have two different versions with published endpoints or the same version on two different endpoints. Once the testing of the staging version pass your test criteria, you can promote/deploy to Production and can ensure the consistency.

Upvotes: 1

Related Questions