CODEWITHSUNDEEP
r
Mayur
Mayur

Reputation: 3

R - reading table from a web page

I want to access the table with list of securities from the below mentioned web page.

https://www.nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm

I am relatively new to R, with the solutions provided in this forum I tried readLines() method, tried using the XML library i.e. readHTMLTable() method as well but couldn't access the table from the web page. Further I also tried using GET() using Chrome as a user_agent(since I was getting forbidden error message). It would be great if someone can look into this.

Thanks in advance :)

Upvotes: 0

Views: 89

Answers (2)

Sathish
Sathish

Reputation: 12723

Your weblink is not pointing to the right location. If you are using mozilla firefox, go to the Developer section and under Network - HTML, you will see all downloaded html pages. If not, reload the page. One of them is the correct one. I have included it in the following code.

library("httr")
URL <- "https://www.nseindia.com/products/content/derivatives/equities/fo_underlyinglist.htm"
temp <- tempfile(fileext = ".html")
GET(url = URL, user_agent("Mozilla/5.0"), write_disk(temp))

library("XML")
df <- readHTMLTable(temp)
df <- df[[1]]

> head(df)
  S. No.\n    Underlying\n     Symbol
1        1       INDIA VIX   INDIAVIX
2        2        Nifty 50      NIFTY
3        3        Nifty IT    NIFTYIT
4        4      Nifty Bank  BANKNIFTY
5        5 Nifty Midcap 50 NIFTYMID50
6        6       Nifty PSE   NIFTYPSE

Upvotes: 2

shayaa
shayaa

Reputation: 2797

This should get you started 

library(httr)
site <- GET("https://www.nseindia.com/products/content/derivatives/equities/fo_underlying_home.htm",
         user_agent("Mozilla/5.0"))
content <- content(site, as="text")
parsedHTML = htmlParse(content, asText = TRUE)

I inspected the element of the table and I copied the id, and entered it into xpathSApply.

xpathSApply(parsedHTML,"//*[@id=\"replacetext\"]/table", xmlValue)

I suspect that there is an encoding problem with UTF-8, but I am no expert in xpath.

Upvotes: 0

Related Questions