rvest: Scraping table from webpage

Question

I am trying to retrieve the following table:

I managed to retrieve the quotes using the following code:

library('rvest')
url.2 <- "https://www.wettportal.com/Fussball/Champions_League/Champions_League/Paris_Saint-Germain_-_Real_Madrid_2448367.html"
webpage.2 <- read_html(url.2)
oddscell.html <- html_nodes(webpage.2, ".oddscell")
oddscell.data <- html_text(oddscell.html)
home <- oddscell.data[seq(1, length(oddscell.data), 3)]
draw <- oddscell.data[seq(2, length(oddscell.data), 3)]
away <- oddscell.data[seq(3, length(oddscell.data), 3)]

my.quotes <- cbind(home, draw, away)

With the following result (only the first 3 rows):

  my.quotes[1:3,]
  home                     draw                    away                   
 [1,] "1.67"                   "4.25"                  "4.35"                 
 [2,] "1.68"                   "4.10"                  "4.20"                 
 [3,] "1.72"                   "4.70"                  "4.56"

I managed to do something similar to retrieve the name of the bookies using html_nodes(webpage.2, ".bookie").

My question is: Is there a way to scrape the table all at once?

ASH · Accepted Answer

That site is blocked for me! I can't see anything there, but I can tell you, basically, it should be done like this.

The html_nodes() function turns each HTML tag into a row in an R dataframe.

library(rvest)

## Loading required package: xml2

# Define the url once.
URL <- "https://scistarter.com/finder?phrase=&lat=&lng=&activity=At%20the%20beach&topic=&search_filters=&search_audience=&page=1#view-projects"

    scistarter_html <- read_html(URL)
    scistarter_html

## {xml_document}
## 
## [1]



We’re able to retrieve the same HTML code we saw in our browser. This is not useful yet, but it does show that we’re able to retrieve the same HTML code we saw in our browser. Now we will begin filtering through the HTML to find the data we’re after.

The data we want are stored in a table, which we can tell by looking at the “Inspect Element” window.

This grabs all the nodes that have links in them.

    scistarter_html %>%
      html_nodes("a") %>%
      head()

## {xml_nodeset (6)}
## [1] My Account
## [3] Project Finder
## [4] Event Finder
## [5] People Finder
## [6] log in


In a more complex example, we could use this to “crawl” the page, but that’s for another day.

Every div on the page:

    scistarter_html %>%
      html_nodes("div") %>%
      head()

## {xml_nodeset (6)}
## [1] 
        
          < ...
## [3] 
             ...
## [4] 
              
 ...
## [5] 
                
                    


… the nav-tools div. This calls by css where class=nav-tools.

    scistarter_html %>%
      html_nodes("div.nav-tools") %>%
      head()

## {xml_nodeset (1)}
## [1] 
             ...


We can call the nodes by id as follows.

    scistarter_html %>%
      html_nodes("div#project-listing") %>%
      head()

## {xml_nodeset (1)}
## [1] 
          
       ...


All the tables as follows:

    scistarter_html %>%
      html_nodes("table") %>%
      head()

## {xml_nodeset (6)}
## [1] 




https://rpubs.com/Radcliffe/superbowl

rvest: Scraping table from webpage

Answers (1)

Related Questions