CODEWITHSUNDEEP
pythonhtmlweb-scraping
yashaswi kashyap
yashaswi kashyap

Reputation: 37

Scaping data from webpage

I am trying to scrape data from this web page: I finally reached this link from HTML where the data is stored, I just not able to read data using my code. can someone tell me what I am doing wrong?
https://www.moneycontrol.com/stocks/fno/query_tool/get_expdate.php?symbol=ACC&inst_type=FUTSTK

import requests
import pandas as pd
url ="https://www.moneycontrol.com/stocks/fno/query_tool/get_expdate.php?symbol=ACC&inst_type=FUTSTK"
headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) chrome/80.0.3987.132 Safari/537.36','Accept-Language': 'en-US,en;q=0.9','Accept-Encoding': 'gzip, deflate'}
response = requests.get(url, headers=headers)
response.raise_for_status()
data = response.json()["data"]

Essentially, questions will boil down to how to read this data from the PHP page?

Upvotes: 0

Views: 58

Answers (2)

Juan Carlos Villalobos
Juan Carlos Villalobos

Reputation: 55

the page does return a json?, try

...

response = requests.get(url, headers=headers)

print(response.content)

Upvotes: 0

Andrej Kesely
Andrej Kesely

Reputation: 195438

The data from the URL is not in Json format, but HTML code.

You can parse it with this script:

import requests
from bs4 import BeautifulSoup

url ="https://www.moneycontrol.com/stocks/fno/query_tool/get_expdate.php?symbol=ACC&inst_type=FUTSTK"
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

for o in soup.select('option'):
    print(o.text)

Prints:

30-07-2020
27-08-2020
24-09-2020

Upvotes: 1

Related Questions