CODEWITHSUNDEEP
pythonweb-scrapingdata-science
Ross Leavitt
Ross Leavitt

Reputation: 149

Why is my web scraping producing HTML but won't return any text?

New coder here. I am trying to return all the earnings per share data from this website here: https://www.nasdaq.com/market-activity/stocks/csco/revenue-eps

I started off slow by just trying to return "March", and used this code: 

from bs4 import BeautifulSoup
from requests import get

url = "https://www.nasdaq.com/market-activity/stocks/csco/revenue-eps"
response = get(url)
soup = BeautifulSoup(response.text, 'html.parser')

month = soup.find("th", {"class": "revenue-eps__cell revenue-eps__cell--rowheading"})

print(month.text)

When I run it there are no errors, but nothing is returned.
When I try running the same code but use print(month) instead, I return the HTML from the element that looks like the following:
th class="revenue-eps__cell revenue-eps__cell--rowheading" scope="row"> /th>

I noticed in the HTML that is returned, that the text isn't inside the th. Why is that? Am I doing something wrong or is it the site I'm trying to scrape?

Upvotes: 0

Views: 178

Answers (1)

Bertrand Martel
Bertrand Martel

Reputation: 45513

The data is not embedded in the page but retrieved from an API. You can pass the company name as parameter to get all the data directly : 

import requests
import json

company = "CSCO"
r = requests.get("https://api.nasdaq.com/api/company/{}/revenue?limit=1".format(company))

print(json.loads(r.text)['data'])

Upvotes: 1

Related Questions