CODEWITHSUNDEEP
rweb-scrapingrvest
user86907
user86907

Reputation: 837

div class scraping

I am trying to scrape a table from the following web using the code below:

library(rvest)
library(tidyverse)
library(dplyr)

base<-'******************'

links<-read_html(base)%>%html_nodes(".v-data-table__wrapper")

But no luck yet. Can anyone help me with this please?

Upvotes: 0

Views: 163

Answers (1)

xwhitelight
xwhitelight

Reputation: 1579

There's no table in the page source originally. This page uses JS to generate the table: enter image description here

The idea is to run the JS code to get the data (you will need the V8 package):

library(V8)
library(rvest)
js <- read_html("https://www.locate.ai/retail-tracker.html") %>%
  html_node(xpath = "//script[contains(., 'gon.data')]") %>% html_text()
ct <- V8::new_context()
ct$eval("var window = {}, gon = {};") # need to initialize variables first
ct$eval(js)
data <- ct$get("gon")
# mining the data
cities <- data$regions
retailbrands <- data$brands

Results:

> head(cities)
           region     change
1 Minneapolis, MN -0.7164120
2      Boston, MA -0.6337319
3  Washington, DC -0.6191386
4     Detroit, MI -0.5693641
5     Chicago, IL -0.5101856
6   Charlotte, NC -0.4810490

> head(retailbrands)
            brand     change
1      LA Fitness -0.6168534
2     Wells Fargo -0.5355715
3     Foot Locker -0.5211365
4     Ethan Allen -0.5096331
5     Clean Juice -0.5079978
6 Texas Roadhouse -0.4770344

Upvotes: 4

Related Questions