CODEWITHSUNDEEP
r
user8959427
user8959427

Reputation: 2067

Scraping a table from a website using rvest

I am trying to scrape the table from the treasury website.

https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldYear&year=2019

What I have currently is to collect the data but 

library("rvest")
url <- "https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldAll"

data <- url %>%
  html()

But I cannot seem to get it into a table format since I have a function.

data %>%
html_table()

Upvotes: 0

Views: 86

Answers (1)

Yifu Yan
Yifu Yan

Reputation: 6106

It's better to first use CSS to locate the node which contains the table. The table is big(around 7400 rows). It took 30 seconds to render using html_table.

library("rvest")
library(httr)
url <- "https://www.treasury.gov/resource-center/data-chart-center/interest-rates/Pages/TextView.aspx?data=yieldAll"
ua <- "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/77.0.3865.90 Safari/537.36"

data <- html_session(url,user_agent(ua))
data %>%
    html_node("table.t-chart") %>%
    html_table()

Upvotes: 1

Related Questions