CODEWITHSUNDEEP
pythonselenium
Volatil3
Volatil3

Reputation: 15016

Python Selenium: Unable to fetch table content

I am trying to access this URL, here I have to fetch table under Price / Tax History section. Below is my code:

from selenium import webdriver
from selenium.webdriver.common.by import By

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from time import sleep
import os, sys
from multiprocessing import Pool
from selenium.webdriver import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait

driver = webdriver.Firefox()
wait = WebDriverWait(driver, 5)
driver.maximize_window()
driver.get('https://www.zillow.com/homedetails/2114-Bigelow-Ave-N-Seattle-WA-98109/48749425_zpid/')
sleep(10)
p_history = driver.find_elements_by_css_selector('#tax-price-history  table tr > td')
    for p in p_history:
        print(p.text)

it is not printing text.

Update Screen of the section required:

enter image description here

Update#2

Ran against PhantomJS and here you can see loader image in the section(Scroll the image)

enter image description here

Upvotes: 0

Views: 192

Answers (1)

jymbob
jymbob

Reputation: 488

You need to tell selenium to use WebDriverWait and expected_conditions to find the element once it's loaded.

You need a reference to an element that doesn't exist on page load, but should be there once the Ajax request is complete. It looks like #tax-price-history table should fulfil that requirement.

try:

from selenium.webdriver.support import expected_conditions as EC
parent = wait.until(EC.presence_of_element_located((
    By.CSS_SELECTOR, '#tax-price-history table')))

p_history = parent.find_element_by_css_selector('td')

If the element isn't found in the time limit specified in wait you'll get an error

Upvotes: 2

Related Questions