CODEWITHSUNDEEP
rdplyrrvest
papelr
papelr

Reputation: 438

Scraping with rvest not working properly

I want to scrape the second main table on this page: https://www.hockey-reference.com/players/f/forsbfi01.html, the "NHL Possession Metrics" table. Rvest should allow me to do this easily:

fil_link <- "https://www.hockey-reference.com/players/f/forsbfi01.html"

fil_pos <- fil_link %>% 
 read_html %>% 
 html_node(css = "#skaters_advanced") %>% 
 html_table(header = T)

But I get this error:

Error in UseMethod("html_table") : 
  no applicable method for 'html_table' applied to an object of class 
  "xml_missing"

I got the first table with essentially the same code, while swapping out the css code. Plus, I've looked around the table code, checking to make sure I get the actual table selector, and not the container, innards, etc. Why won't this scrape?

Upvotes: 1

Views: 1863

Answers (1)

Nicol&#225;s Velasquez
Nicol&#225;s Velasquez

Reputation: 5898

This would solve the problem for this particular table. But please note that it is a very fragile solution. It assumes a matrix structure of only 3 rows, and expects all rows to have the same number of elements.

library(tidyverse)
library(rvest)

## Create a function to parse every row in the table.
Hockey_table <- function(htmlObject) { 

  titlerow <- htmlObject %>% html_nodes(xpath = '//div[@class="stats_pullout"]/div/div/h4') %>%  html_text('data-tip')
  firstrow <- htmlObject %>% html_nodes(xpath = '//div[@class="stats_pullout"]/div/div/p[1]') %>% html_text()
  secondrow <- htmlObject %>% html_nodes(xpath = '//div[@class="stats_pullout"]/div/div/p[2]') %>% html_text()

  data.frame(titlerow, firstrow, secondrow)

}

 Page01 <- read_html('https://www.hockey-reference.com/players/f/forsbfi01.html')
Hockey_table(Page01)

Results:

   titlerow firstrow secondrow
 1   SUMMARY  2017-18    Career
 2        GP       67       331
 3         G       26       117
 4         A       38       138
 5       PTS       64       255
 6       +/-       27        26
 7        PS      8.3      32.1
 8       PIM       38       145
 9        SH      179       931
 10      GWG        6        24
 11      TOI    17:28     17:49
 12      CF%     53.0      54.9
 13     oZS%     54.1      63.4

Upvotes: 1

Related Questions