CODEWITHSUNDEEP
rweb-scraping
naro
naro

Reputation: 105

R - Web scraping item price

I'm trying to write an R script checking prices on a popular swiss website.

Following methodology explained here: https://www.analyticsvidhya.com/blog/2017/03/beginners-guide-on-web-scraping-in-r-using-rvest-with-hands-on-knowledge/ I tried to use rvestfor that:

library(rvest)

url <- "https://www.galaxus.ch/fr/s8/product/quiksilver-everyday-stretch-l-shorts-de-bain-10246344"

webpage <- read_html(url)

Unfortunately, I have limited html/css knowledge and the content of webpage is very obscure to me.

I tried inspecting the page with google chrome and it looks like the price is located in something named priceEnergyWrapper--2ZNIJ but I cannot find any trace of that in webpage. I did not have more luck using SelectorGadget priceEnergyWrapper--2ZNIJ

Can anybody help me get the price out of webpage?

Upvotes: 1

Views: 840

Answers (1)

Ankur Sinha
Ankur Sinha

Reputation: 6639

Since it is dynamically generated, you will need RSelenium.

Your code should be something like:

library(RSelenium)

driver <- rsDriver(browser=c("chrome"))
rem_driver <- driver[["client"]]
rem_driver$open()
rem_driver$navigate("https://www.galaxus.ch/fr/s8/product/quiksilver-everyday-stretch-l-shorts-de-bain-10246344")

This will ask Selenium to open this page after loading the entire page, and hence all the HTML that you see by clicking Page Source should be available.

Now do:

rem_driver$findElement(using = 'class', value = 'priceEnergyWrapper--2ZNIJ')

You should now see the necessary HTML to get the price value out of it, which at the time of checking the website is 25 CHF.

PS: I do not scrape websites for others unless I am sure that the owners of the websites do not object to crawlers/scrapers/bots. Hence, my codes are based on the idea of how to go about with Selenium. I have not tested them personally. However, you should more or less get the general idea and the reason behind using a tool like Selenium. You should also find out if you are allowed to legally scrape this website and for others in the near future.

Additional resources to read about RSelenium: https://ropensci.org/tutorials/rselenium_tutorial/

Upvotes: 3

Related Questions