CODEWITHSUNDEEP
pythonpython-2.7seleniumphantomjs
Lawrence Muriuki
Lawrence Muriuki

Reputation: 136

Selenium and phantomjs not opening javascript page

Am trying to access odds data contained in a javascript rendered page but selenium doesn't seem to work. Here's my code:

from selenium import webdriver

browser = webdriver.PhantomJS()
browser.implicitly_wait(10)
browser.get('http://justbet.co.ke/index.php?option=com_jbt3&task=matches&league=58&Itemid=1')
print browser.page_source

Any help will be appreciated

Upvotes: 3

Views: 2754

Answers (1)

alecxe
alecxe

Reputation: 474191

You just need to wait for the main content to load:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".razrada")))

Complete working code using this page as a target:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Firefox()
driver.get("http://justbet.co.ke/index.php?option=com_justbet&league=58&Itemid=1")

# wait for the page to load
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".razrada")))

for row in driver.find_elements_by_css_selector(".razrada > tbody > tr"):
    print([td.text.replace("\n", " ")
           for td in row.find_elements_by_css_selector("table.options td.option")])

Prints:

['Crystal Palace 2.95', 'X 3.20', 'Everton FC 2.40']

Upvotes: 2

Related Questions