CODEWITHSUNDEEP
pythonhtmlweb-scraping
yash
yash

Reputation: 27

Scraping html table using python

I am trying to scrape a table through from a website but i am getting NULL.

How can i get the table? What am i doing wrong?

import requests
from bs4 import BeautifulSoup

html = "https://traderslounge.in/implied-volatility-rank-nse-fno-stocks/" #link that has to be scrapped

response = requests.get(url) # before we feed it to request to parse 

response.status_code
soup = BeautifulSoup(response.text, 'html.parser')

table = soup.find_all("th")
list_of_rows = []
for row in table.findAll("td"):
    list_of_cells = []
    for cell in row.findAll(["th","td"]):
        text = cell.text
        print(text)
        list_of_cells.append(text)
        list_of_rows.append(list_of_cells)

for item in list_of_rows:
    print(' '.join(item))

Upvotes: 1

Views: 90

Answers (1)

Bertrand Martel
Bertrand Martel

Reputation: 45402

The table content of this site is retrieved from an external API :

https://traderslounge.in/FNO/ivrank/ivranktable.txt

You can get the result using :

import requests

r = requests.get('https://traderslounge.in/FNO/ivrank/ivranktable.txt')

print(r.json()["data"])

Upvotes: 2

Related Questions