wit.ai stories training results unpredictable?

Question

I am trying to develop a weather bot and have been experiencing some problems.

1. I trained the systems with stories and in the understanding tab. The behavior of wit seems unpredictable to me - it combines stories (which it is supposed to do I guess), so seems converse randomly.
2. The intent values are not consistent either, even using the same story. I used a debugger to show the intent value. For example, I say "What's the weather", the trained intent value should be forecast_all, but sometimes it becomes "what", and sometimes some other entities from other stories shows up such as "off_topic" entity I created for off topic conversations. Then the bot will behave differently from my training story.

Any insight? Am I doing something wrong? What did I miss? Thank you very much for your help!

Answer 1

I wanted to clarify an importan concept here. The context is yours. Wit only reads it to make prediction of the next step or to generate a Bot answer via variable. We never update the context on our side.

You would use action on your side to update keys in the context. These keys can in turn influence the prediction based on the stories. Wit use the current context (keys only for now), the previous states of the context (at interaction N-1, N-2….) but also the user message (trait entities) to predict the next action.

Additional info here: https://wit.ai/docs/recipes#manage-context-and-session-id

Answer 2

I feel your pain:)

You have to "force" wit.ai into correct stories based on the user input and if you need the user to follow the story till the end - you will have to keep "forcing" them down the dialogue chain.

What I mean is this:

If user says What's the weather you can introduce one more custom entity by the name wth and in the Understanding tab make it keywords based and add strict keywords list related to the weather word (forecast...).

This could be your "guiding thread". When this entity appears = user has used a keyword associated with the weather question = you can use that entity wth as a requirement on every action and response of your story (Actions tab).

In a few apps I have - for each story I thought of "what is this story doing" - what's the entry point, so to say and came up with an entity which would lead the user down that dialogue path by using Actions tab and setting Only if...Always if... requirements on replies and actions.

This approach is very strict, but it gives you some decent level of control over what user receives from your "bot" so you can anticipate actions and replies.

I was playing around with another approach where you have a lot of stories, they are all "incomplete" but they are trying to cover possible different answers from the user. So user starts it's chatting in one story, then depending on his answer - another story kicks in, then another, then another...

Also a valid, more flexible approach, but it needs a very good planning of the "bot replies" \ stories so that they feel connected in a logical, conversational way. And yes it also needs a careful context management because if every story will add context - it might become a big pile of entities and values, so you will need to think that story A adds entity A1 to the context, but if later a story B kicks in and adds B1 entity to the context then I need to remove A1 because they are conflicting...

To summarize: It seems like you want to have a bit more control and know where the dialogue goes. For that - try the Actions tab and setting up requirements for answers and actions. You can make it very straight forward and direct.