CODEWITHSUNDEEP
rweb-scraping
Mark P.
Mark P.

Reputation: 1827

Scrape data using R when div class is different for the same attribute

I am trying to extract a full description from a website with a div class as such:

enter image description here

This div class changes for data that would go in the same column. I am using the following r code for other div classes that do not change:

#get the beer IBU
num_ibu <- html_nodes(webpage, ".ibu")
num_ibu <- as.character(html_text(num_ibu))

My question is how do I modify this code to find a div class like '.desc-full'? I have tried full_desc <- html_nodes(webpage, ".desc-full*") only to receive the following error:

Error in parse_simple_selector(stream) : 
  Expected selector, got <DELIM '*' at 11>

I seem to be having a difficult time finding a like command that works in html_nodes. Is this a case where I should use regex? That feels like overkill.

Upvotes: 2

Views: 633

Answers (1)

J. Win.
J. Win.

Reputation: 6771

webpage <- "https://untappd.com/beer/top_rated?country_id=86"

sess <- html_session(webpage)
all_desc_nodes <- html_nodes(sess, ".desc")
full_desc_nodes <- all_desc[grep("desc-full", all_desc_nodes)]
full_desc_text <- html_text(full_desc_nodes)

Upvotes: 1

Related Questions