CODEWITHSUNDEEP
web-scrapingbeautifulsouppython-requestsdynamic-data
Seth Bowers
Seth Bowers

Reputation: 35

Pulling Yahoo Finance Quotes Using Requests / BS4

GOAL:

The end goal is to be able to retrieve the dynamic data found in the table under the 'Summary' tab on a Yahoo Finance page. I need to be able to do this without the use of a 3rd party API like yahoo_fin or yfinance. To clarify which table (there are many), here is a screenshot:

enter image description here

(Link to the Exact Webpage).

The only cells I really need are the 'Volume' and 'Avg. Volume' values, but I believe these are only available via an HTML element of the following structure:

enter image description here

Code:

import requests
from bs4 import BeautifulSoup

url = 'https://finance.yahoo.com/quote/AMZN?p=EDTK&.tsrc=fin-srch'
r = requests.get(url)

web_content = BeautifulSoup(r.text,'lxml')
table_values = web_content.find_all('div', class_='Bxz(bb) Bdbw(1px) Bdbs(s) Bdc($seperatorColor) H(36px)')

print(table_values)

returns: '[]'

Is there anything I'm clearly doing wrong? I'm no expert in bs4, but I've used this syntax before in the past and never had any issues.

Upvotes: 1

Views: 651

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195553

You're selecting <div> instead of <tr>:

import requests
from bs4 import BeautifulSoup

url = "https://finance.yahoo.com/quote/AMZN?p=EDTK&.tsrc=fin-srch"
r = requests.get(url)

web_content = BeautifulSoup(r.text, "lxml")
table_values = web_content.find_all(
    "tr", class_="Bxz(bb) Bdbw(1px) Bdbs(s) Bdc($seperatorColor) H(36px)"
)

for row in table_values:
    tds = [td.get_text(strip=True) for td in row.select("td")]
    print(*tds)

Prints:

Previous Close 3,421.57
Open 3,424.80
Bid 3,472.00 x 1000
Ask 3,473.00 x 1100
Day's Range 3,395.59 - 3,472.58
52 Week Range 2,871.00 - 3,773.08
Volume 4,312,055
Market Cap 1.758T
Beta (5Y Monthly) 1.14
PE Ratio (TTM) 60.47
EPS (TTM) 57.40
Earnings Date Oct 27, 2021-Nov 01, 2021
Forward Dividend & Yield N/A (N/A)
Ex-Dividend Date N/A

Upvotes: 2

Related Questions