CODEWITHSUNDEEP
pythongoogle-cloud-platformdialogflow-es
Neurus
Neurus

Reputation: 887

Dialogflow: Intent training phrases - how to include misspelled / informal words

I am building my first agent with DialogFlow, and the intent page has the entry for training phrases. I am testing with a simple intent "greeting". With normal, properly spelled words works just fine, e.g.

Now, if I run DetectIntentText with words like "Hellooooooooo" (9 o's) ... the detected intent gives the Default Fallback intent. So if I add "Hellooooooooo" (with exactly 9 o's) to the training phrases, then the intent is "greeting". But then if I test it with Helloooo (4 o's) the intent given is the Default Fallback intent again ...

I can't predict how many o's the user will put, and it will be cumbersome to add in the training phrases all combinations (e.g. Helloooo, Heeellooooo, etc. etc.). With all the slang going around, and people typing pretty fast, what is the recommended approach? spell check / correct it before sending it to the Dialogflow API? Wouldn't the DialogFlow API catch these?

Thanks!

Upvotes: 0

Views: 1359

Answers (2)

Fcojavmelo
Fcojavmelo

Reputation: 376

As you've mentioned, you'd need to check spelling before sending the request. Your example can work.

Regarding training phrases, the more phrases you add, the more training the agent has to get the context from certain user inputs.

For example, if you had "hello" or "hi" as some of your training phrases, you could send "hey" (not in your training phrases) as request and the intent will still be matched.

However, this only works with phrase context and doesn't by-pass spelling errors. For example, "helloo" wouldn't match that intent. So it's still necessary to check spelling before sending the request.

Alternatively, you could turn on automatic spell correction on the agent’s ML settings.

Warnings and best practices about automatic spell correction:

  • Spell correction can't correct ASR (automatic speech recognition) errors, so we don't recommend enabling it for agents using ASR inputs.
  • It is possible for corrected input to match the wrong intent. You can fix this by adding commonly mismatched phrases to negative examples.
  • Spell correction increases the agent's response time slightly.
  • Spell correction should not be used with Actions on Google.
  • Spell correction is trained on general user queries. If an agent is defined using domain-specific jargon, the corrections may be undesired.

Upvotes: 1

Neurus
Neurus

Reputation: 887

Found out that DialogFlow does not handle typos, it is recommended to do spelling check before sending it to the API.

So in my specific case, what I can do is:

  • Remove all repeated characters:
    word = re.sub(r"(\w)\1*", r'\1', 'heelloooo')
  • Spell check it:
    import pyspellchecker
    from spellchecker import SpellChecker
    spell = SpellChecker()
    corrected = spell.correction(word)

What do you think?

Upvotes: 0

Related Questions