Peter Chung
Peter Chung

Reputation: 1122

R web scraping by class

I have a finance assignment for collecting the beta value for calculation, I am new in R, I would like to web scraping the beta value by package rvest or httr. However, output is character(0).

enter image description here

xpath:
//*[@id="StkList"]/ul/li[48]

library(rvest)
library(dplyr)

sym <- "1212"
url.3 < paste("http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=",sym,sep="")

beta.value <- url.3 %>% read_html() %>% html_nodes(xpath = "//*[@id='StkList']/ul/li[48]")

output:
character(0)

desired output:
0.270

I tried not using xpath but, the html_nodes("div.value.highlight") but don't work as well. Is anyone can help or advice ? Thank you.

Upvotes: 0

Views: 496

Answers (1)

hrbrmstr
hrbrmstr

Reputation: 78792

They check referer before displaying the page, so you have to add some headers:

library(magrittr)
library(httr)
library(rvest)

httr::GET(
  url = "http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=1212", 
  httr::add_headers(
    Host = "www.etnet.com.hk",
    Referer = "http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=1212"
  )
) -> res

res <- content(res, encoding="UTF-8")

html_node(res, xpath=".//li[contains(., 'Beta')]/following-sibling::li[1]") %>% 
  html_text()
## [1] "+0.270"

Upvotes: 2

Related Questions