CODEWITHSUNDEEP
pythonhtmldateweb-scrapingbeautifulsoup
Steve.Kim
Steve.Kim

Reputation: 71

How to scrape the date part from Python

I am trying to web scrape this website: https://www.reuters.com/companies/tsla.oq/financials/income-statement-quarterly

I am using Python and everything can be scraped except the date part... i.e. I can't scrape '30-Jun-20'. I tried like

from requests import get
from bs4 import BeautifulSoup
url = 'https://www.reuters.com/companies/tsla.oq/financials/income-statement-quarterly'
response = get(url)
html_soup = BeautifulSoup(response.text, 'html.parser')
table = html_soup.find_all('div', class_ = 'tables-container')
table[0].thead.tr.find('time', class_ = 'TextLabel__text-label___3oCVw TextLabel__black___2FN-Z TextLabel__medium___t9PWg').text

But it shows blank... Can you please help me? That would be much appreciated.

Upvotes: 0

Views: 207

Answers (1)

Yash
Yash

Reputation: 1281

You cannot get the data from a website using requests whose data is dynamically added (using javascript). You need to use selenium to achieve that.

refer this code:

from selenium import webdriver
from bs4 import BeautifulSoup
DRIVER_PATH="Your selenium chrome driver path"
url = 'https://www.reuters.com/companies/tsla.oq/financials/income-statement-quarterly'
driver = webdriver.Chrome(executable_path=DRIVER_PATH)
driver.get(url)
html_soup = BeautifulSoup(driver.page_source, 'html.parser')
table = html_soup.find_all('div', class_ = 'tables-container')
driver.quit()
print(table[0].thead.tr.find('time', class_ = 'TextLabel__text-label___3oCVw TextLabel__black___2FN-Z TextLabel__medium___t9PWg').text)

Upvotes: 2

Related Questions