CODEWITHSUNDEEP
node.jselasticsearchtensorflow
Hafez
Hafez

Reputation: 1440

tensorflow for classification of strings vs elasticsearch

So, a little bit on my problem.

TL;DR

Can I use machine-learning instead of Elastic Search to find results depending on the user's text input? Is it a good idea?

I am working on a car spare parts project, and we have split the car into 300 parts that we store on the database, with some data for each part (weight, availability, etc).

When the customer inputs the text of his part, we need to be able to classify the part, and map it to one in our database.

The current way it's being done is by people on our team manually mapping the customer text with the parts on our database, we want to automate that process.

We tried using MongoDB text search, but it was often inaccurate since parts have different names in different parts of the country.

So we wanted something that got more accurate results, and improved by the more data we have, we immediately considered TensorFlow, after some research and taking part of Google's Machine Learning Crash Course, I got to that point where it specified:

Models can't learn from string values, so you'll have to perform some feature engineering to convert those values to something numeric

That would be useful in the case we have limited number of features as strings, but we don't know what the user will input as a text.

So, my questions are:

1- Can we use Machine Learning to map text input by the user with some documents on our database?

2- If we can do that, is it a good idea to favor it over other search tools like ElasticSearch?

3- Can ElasticSearch improve its results the more data we have? How?

4- How would you go about this problem?

Note: I'd be doing that in Node.js, and since TensorFlow.js is new, I am inclining to go for other solutions, but if push comes to shove, and the results are much better, I would definitely go there.

Upvotes: 2

Views: 1054

Answers (1)

Peter Szoldan
Peter Szoldan

Reputation: 4868

TL;DR: Yes and yes.

TS;WM: This is a perfectly suited problem for machine learning. Especially so, if you have a database of past customer texts that have already been mapped to parts. Ideally, you have hundreds of texts mapped to each part. If that is present, you can design and train a network. And models can learn from string values with some engineering, and it's not that bad.

I'm not sure ElasticSearch would improve much on the network. I don't know much about auto parts trading, but as a wild guess, "the large round thingy that helps change direction" would never be mapped to "steering wheel" by ES but could be learned easily by a network - provided there are at least some examples of people using that text to specify steering wheel.

You can but don't have to necessarily use tensorflow.js for your network. The AI could run on your server as a webservice, and you'd just send over the customer's text to it and it would send back it's recommendations of part SKUs and names.

Upvotes: 1

Related Questions