CODEWITHSUNDEEP
rscreen-scrapingrvest
Andrzej Szymkowiak
Andrzej Szymkowiak

Reputation: 1

Scraping using rvest

I'm trying to scrape this page https://www.kickstarter.com/projects/883386176/obsbot-tail-the-worlds-first-auto-director-ai-came/rewards

More specifically, the values displayed in the floating menu (number of comments, FAQ, update), but it seems it is not working. Instead of the desired value, I have an empty character.

page source -> https://i.sstatic.net/i4j5r.jpg

Rvest code:

page = "https://www.kickstarter.com/projects/883386176/obsbot-tail-the-worlds-first-auto-director-ai-came/rewards"
page = read_html(page)
page %>% html_nodes(xpath='//*[@class="js-load-project-comments"]') %>% xml_attr("value")

character(0)

But then I analyzed the content [read_html(page)] and I was unable to find this code even manually.

Where did I make a mistake and how to get the value?

Upvotes: 0

Views: 59

Answers (2)

Emmanuel Hamel
Emmanuel Hamel

Reputation: 2213

Here is another approach that can be considered :

library(RDCOMClient)

url <- "https://www.kickstarter.com/projects/883386176/obsbot-tail-the-worlds-first-auto-director-ai-came/rewards"
IEApp <- COMCreate("InternetExplorer.Application")
IEApp[['Visible']] <- TRUE
IEApp$Navigate(url)
Sys.sleep(5)
doc <- IEApp$Document()

web_Obj_Comments <- doc$querySelector("#comments-emoji")
web_Obj_Comments$innerText()

[1] " Commentaires 827

Upvotes: 0

Emmanuel Hamel
Emmanuel Hamel

Reputation: 2213

Here is an approach that can be considered :

library(RSelenium)

port <- as.integer(4444L + rpois(lambda = 1000, 1))
rd <- rsDriver(chromever = "105.0.5195.52", browser = "chrome", port = port)
remDr <- rd$client

url <- "https://www.kickstarter.com/projects/883386176/obsbot-tail-the-worlds-first-auto-director-ai-came/rewards"
remDr$navigate(url)

web_Obj_Comments <- remDr$findElement("xpath", '//*[@id="comments-emoji"]')
web_Obj_Comments$getElementText()

[[1]]
[1] "Commentaires 827"

Upvotes: 0

Related Questions