CODEWITHSUNDEEP
machine-learningnlpldatopic-modeling
sravani.s
sravani.s

Reputation: 185

How to get intent of a document using LDA or any Topic Modeling Algorithm

I need to cluster a group of documents based on the intent they have and I am planning to use LDA(Latent Dirichlet Allocation - Topic Modeling).

Can i get intents to group the documents from topic modeling ? are there any other algorithms that cluster the documents based on the intents they have. Is this approach of using topic-modeling for intent clustering is good ?

I have been trying LDA Algorithm in topic modeling and able to get list of topics but not sure whether i can consider topics as intents itself.

Expecting an approach which clusters the group of documents based on the intents they have.

Upvotes: 0

Views: 527

Answers (1)

DBaker
DBaker

Reputation: 2139

as stated here LDA disregards the structure of how words interact between each other, it will not be suited for intent modeling

As a bag-of-words model is used to represent the documents, LDA can suffer from the same disadvantages as the bag-of-words model. The LDA model learns a document vector that predicts words inside of that document while disregarding any structure or how these words interact on a local level.

Consider the following to sentences:
This is his sister's dog (statement)
Is this his sister's dog (question)

Same words, different order, different intent.

You will probably need labeled data, and the use of neural networks such as CNNs or LSTMs.

Upvotes: 1

Related Questions