CODEWITHSUNDEEP
rlinear-regressionldatopic-modeling
allmine
allmine

Reputation: 385

How to implement Latent Dirichlet Allocation in regression analysis

I have a dataset consisting of hotel reviews, ratings, and other features such as traveler type, and word count of the review. I want to perform topic modeling (LDA) and use the topics derived from the reviews as well as other features to identify the features that most affects the ratings (ratings as the dependent variable).

If I want to use linear regression to do this, does this mean I would have to label each review with the topics derived? Is there a way to do this in R or will I have to manually label each review? (I am new to text mining and data science in general.)

Upvotes: 1

Views: 1498

Answers (1)

user799188
user799188

Reputation: 14425

The short answer : you don't have to label each review with the topics derived because you'd be relying on the topic model you train to determine the topics of the reviews, which would then be used to construct features for your regression model.

There is a good explanation of topic modeling with code samples (in R) at
www.tidytextmining.com/topicmodeling.html. Sections 6.2.1 and 6.2.2 should help you quickly get started.

Keeping in mind the following two principles

  • Every document (hotel review) is a mixture of topics
  • Every topic is a mixture of words

once a topic model has been trained on the reviews, for every review,

  • the Document-topic probabilities could be used as features
  • the top N terms within each topic could be used to construct a Document-Term Matrix (each review mapped with zero or more of the top terms) which could then be used as additional features

A simplified example : there might be 4 topics the reviews broadly fall under.

  • Topic 1 might be about location (top terms : convenient, location, train_station, walk_distance, shopping, etc)
  • Topic 2 might be about hotel staff (top terms : reception, friendly, professional, quick, late_checkout etc.)
  • Topic 3 might be about hotel rooms (top terms : clean_room, decor, tasteful, etc.)
  • Topic 4 might be about hotel amenities (top terms : pool, wifi, fitness_centre, etc.)

The document-topic probabilities combined with the top terms of each topic can be used as features similar to :

  • topic_1_location_probability
  • topic_2_hotel_staff_probability
  • topic_3_hotel_room_probability
  • topic_4_hotel_amenities_probability is_convenient_location
  • is_train_station_nearby
  • is_walk_distance
  • is_clean
  • is_late_checkout
  • is_fitness_centre
  • etc.

For newer reviews :

  • The example above shows how the initial training dataset would be created - based on which you train your models.
  • For newer reviews (i.e. the ones previously not used for training the models) you don't have to repeat the entire exercise above. Instead, a trained topic model can be used to identify topics of previously unseen documents (reviews). Answers to this question has sample code to help do this.

I hope this helps you.

Upvotes: 3

Related Questions