CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
Cihan
Cihan

Reputation: 33

Web scraping stock price - Yahoo Finance

With my code, I'm able to get real time stock price from Yahoo Finance.

My var. 'maks' defines the number of seconds to record live data. This works fine until 2000 seconds (which is roughly 2000 price ticks).

However, when i define longer periods - say 2 hours and more - i receive below error:

from bs4 import BeautifulSoup
import ssl
import sys
import time
from urllib.request import Request, urlopen

# For ignoring SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE

maks = int(input('Enter time to record data (seconds) : '))

#List for collected values
price_list = []
vol_list = []
time_list = []

print("Parsing data, please wait..")
start = time.time()

i = 0
while i < maks:

 # Making the website believe that you are accessing it using a Mozilla browser
 req = Request('http://finance.yahoo.com/quote/BTC-USD', headers={'User-Agent': 'Mozilla/5.0'})

 web_page = urlopen(req).read()

 # Creating a BeautifulSoup object of the HTML page for easy extraction of data.
 soup = BeautifulSoup(web_page, 'html.parser')
 html = soup.prettify('utf-8')

 new_price = soup.find(id="quote-market-notice").find_parent().find("span").text

 #volume
 vol = soup.find('td', attrs={'data-test': 'TD_VOLUME-value'})
 real_vol = vol.find('span', recursive=False)
 current_vol = real_vol.text.strip()

 saat = time.strftime('%c')

 #Saving values in lists
 price_list.append(new_price)
 vol_list.append(current_vol)
 time_list.append(saat)

 i += 1

ERROR CODE:

File "C:/Users/user/PycharmProjects/untitled/trader.py", line 29, in <module>
    # Creating a BeautifulSoup object of the HTML page for easy extraction of data.

File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 563, in error
    result = self._call_chain(*args)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 755, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Users\user\AppData\Local\Programs\Python\Python37\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

urllib.error.HTTPError: HTTP Error 404: Not Found

Upvotes: 1

Views: 2425

Answers (2)

GabrielP
GabrielP

Reputation: 782

I think you might use the yfinance library (import as yf) that connects to the Yahoo Finance API. https://pypi.org/project/yfinance/

yf.Ticker('TICKER').history()

Upvotes: 1

Naapi
Naapi

Reputation: 46

Could be that you are hitting the API call limit as mentioned in other thread answer, which is Hourly Cap 2,000 requests/hour per IP with Public:

What is the query limit on Yahoo's Finance API?

More about it here:

Usage Information and Limits

Upvotes: 3

Related Questions