CODEWITHSUNDEEP
pythonpandasweb-scraping
Divine
Divine

Reputation: 1

Scrape table from webpage into a DataFrame using read_html

I'm trying to do some webscraping but it keeps showing no tables found but there is literally a table on the website. Maybe I'm not writing the right code. Someone help. Here is the link to the website. Film-Locations-in-San-Francisco

error image enter image description here

Upvotes: 0

Views: 59

Answers (1)

keramat
keramat

Reputation: 4543

I guess you are using requests. But you should note that the requests does not receive the table because the table is generated after rendering the page and it is not exists in the requests data. A basic way is to use something like Selenium to first load the page then retrieve the information.

Based on your comment use something similar to:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By

driver = webdriver.Chrome(executable_path=r"C:/Users/user/Desktop/chromedriver_win32/chromedriver.exe")
driver.get("https://data.sfgov.org/Culture-and-Recreation/Film-Locations-in-San-Francisco/yitu-d5am/data")
elem = driver.find_element_by_xpath('//*[@id="renderTypeContainer"]/div[4]/div[2]/div/div[4]/div[1]/div/table')
elem.text
#driver.close()

Upvotes: 1

Related Questions