CODEWITHSUNDEEP
pythonbeautifulsouphtml-parsing
Anand Shukla
Anand Shukla

Reputation: 9

unable to parse a webpage using python

I am trying to parse below webpage to get name of stocks hitting now all time high or low in the exchange.

https://www.bseindia.com/markets/equity/EQReports/HighLow.html?Flag=H#

however, when i download the webpage using beautiful soup and check the data i do not find the stock name or price mentioned in the webpage. I wish to write a function to download the stocks that hit a new all time high each day please help what am i missing?

Upvotes: 0

Views: 353

Answers (1)

gmds
gmds

Reputation: 19885

Part of the HTML on the page is generated dynamically by JavaScript. You are most likely using the requests library, which cannot handle HTML generated in this way.

What you can do, instead, is use the Selenium library, which allows you to launch an instance of a web browser controlled by Python, and get the page source from there.

from selenium import webdriver

path = '...' # path to driver here
url = 'https://www.bseindia.com/markets/equity/EQReports/HighLow.html?Flag=H#'

driver = webdriver.Chrome(path)
page_source = driver.get(url).page_source

By parsing page_source with BeautifulSoup, you can get what you want.

Upvotes: 2

Related Questions