CODEWITHSUNDEEP
rcurl
Savi
Savi

Reputation: 167

scraping issue - need advice

I am trying to scrape below web page (size details)

 parenturl = http://www.newlook.com/shop/womens/jackets-and-coats/navy-aztec-faux-shearling-collar-parka_286764649?tmcampid=UK_AFF_AffiliateWindow

srcpage = getURLContent(GET(parenturl)$url,timeout(10))
page = htmlTreeParse(srcpage,useInternalNodes = T,encoding='UTF-8')

looking at the page structure, i believe its running javascript in background and gets data from server. I am not sure how to scrape this webpage. Any help would be greatly appreciated.

Many Thanks, Savi

Upvotes: 0

Views: 105

Answers (1)

jdharrison
jdharrison

Reputation: 30425

You can look at using Selenium to do this:

require(RSelenium)
RSelenium::startServer()
appURL <- "http://www.newlook.com/shop/womens/jackets-and-coats/navy-aztec-faux-shearling-collar-parka_286764649?tmcampid=UK_AFF_AffiliateWindow"
remDr <- remoteDriver()
remDr$open()
remDr$navigate(appURL)
inventory <- remDr$executeScript("return list;")
> do.call(rbind.data.frame, inventory)
color listPrice popupImage   skuID
2                0            2867684
21               0            2867685
swatchImage largeImage salePrice
2                                 0
21                                0
detailImage stockLevel size
2                      75   12
21                    133   14

remDr$close()
remDr$closeServer()

Upvotes: 3

Related Questions