CODEWITHSUNDEEP
pythonrweb-scraping
Bogaso
Bogaso

Reputation: 3318

Download data from Internet

I am looking for some way (preferably non-Selenium based) to download data from below link

https://www.nseindia.com/circulars/circular.htm

I tried to use the function XML::readHTMLTable() in R but could not succeed. Even when I tried to see the source of this page I dont see relevant information in the source page.

Any pointer to download data either using R or Python will be highly helpful.

Thanks,

Upvotes: 0

Views: 83

Answers (1)

Yifu Yan
Yifu Yan

Reputation: 6116

The table you're looking at is not in the page source. Your browser runs JavaScript and makes additional request to get this table.

You can look up the actual URL of that table using your developer tools, it's inside the network/xhr tab.

library(rvest)
library(httr)
url <- "https://www.nseindia.com/circulars/content/circ_latest.htm"
ua <- "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
response <- httr::GET(url,user_agent(ua))
html_table(content(response))

enter image description here

For your reference, you can read this section:

https://github.com/yusuzech/r-web-scraping-cheat-sheet#rvest7.1

Upvotes: 2

Related Questions