CODEWITHSUNDEEP
machine-learningtext-classification
sissonb
sissonb

Reputation: 3780

Is text classification fast enough for type ahead search?

I'm working on designing a typeahead service that can be used to search for many different things. I was thinking about creating a text classification model to categorize these searches before actually making the search.

Here's an example of the result I'd want from the classification model.

Input

John Smith

Output

[
  {
    "likeliness": .6,
    "category": "car-name-typeahead-search"
  },
  {
    "likeliness": .9,
    "category": "person-name-typeahead-search"
  },
  {
    "likeliness": .1,
    "category": "vin-typeahead-search"
  },
  {
    "likeliness": .2,
    "category": "help-page"
  },
  {
    "likeliness": .2,
    "category": "faq-page"
  }
]

Then I'd take the categories that have a likeliness above some value and actually do the typeahead search. Also I'd return the results ordered by the likeliness rank.

We have been collecting data about people's searches and tracking what they were actually looking for so we should have the data needed to train a text classification model.

My question is can text classification models be fast enough to be used with a type ahead service and not be prohibitively expensive? Are there certain types of text classification algorithms that I should be looking at?

Upvotes: 0

Views: 164

Answers (1)

greeness
greeness

Reputation: 16104

Usually in modern serving framework (like tensorflow serving running on a standalone server), a standard text classification model based on shallow neural networks should have a latency under 1ms). You can look for a model composed by:

  • word embedding layer (up to millions of words in vocabulary)
  • hidden layer (1-3)
  • classification (up to thousands of categories)

If your expected response time is <= 200ms, you should not worry about the latency from the classification. In the worst case, 10ms is sufficient using the setup above.

Upvotes: 1

Related Questions