Reputation: 1122
I have a finance assignment for collecting the beta value for calculation, I am new in R, I would like to web scraping the beta value by package rvest or httr. However, output is character(0).
xpath:
//*[@id="StkList"]/ul/li[48]
library(rvest)
library(dplyr)
sym <- "1212"
url.3 < paste("http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=",sym,sep="")
beta.value <- url.3 %>% read_html() %>% html_nodes(xpath = "//*[@id='StkList']/ul/li[48]")
output:
character(0)
desired output:
0.270
I tried not using xpath but, the html_nodes("div.value.highlight") but don't work as well. Is anyone can help or advice ? Thank you.
Upvotes: 0
Views: 496
Reputation: 78792
They check referer before displaying the page, so you have to add some headers:
library(magrittr)
library(httr)
library(rvest)
httr::GET(
url = "http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=1212",
httr::add_headers(
Host = "www.etnet.com.hk",
Referer = "http://www.etnet.com.hk/www/eng/stocks/realtime/quote.php?code=1212"
)
) -> res
res <- content(res, encoding="UTF-8")
html_node(res, xpath=".//li[contains(., 'Beta')]/following-sibling::li[1]") %>%
html_text()
## [1] "+0.270"
Upvotes: 2