Howell Yu
Howell Yu

Reputation: 73

Having Trouble Extracting Flight Price Content Using selenium

I tried to extract some flight price information from https://www.google.com/flights/explore but the screenshot I got is blank. Can anyone see what's the problem?

from selenium import webdriver 
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from bs4 import BeautifulSoup

url = "https://www.google.com/flights/explore/#explore;f=JFK,EWR,LGA;t=r-Mexico-0x84043a3b88685353%253A0xed64b4be6b099811;li=3;lx=12;d=2017-05-13"
driver = webdriver.PhantomJS()
dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = (my_agent)
driver = webdriver.PhantomJS(desired_capabilities = dcap,service_args=['--ignore-ssl-errors=true'])
driver.implicitly_wait(20)
driver.get(url)

driver.save_screenshot(r'flight_explorer.png')

Upvotes: 3

Views: 309

Answers (1)

Pavan Kumar T S
Pavan Kumar T S

Reputation: 1559

try this

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://www.google.com/flights/explore/#explore;f=JFK,EWR,LGA;t=r-Mexico-0x84043a3b88685353%253A0xed64b4be6b099811;li=3;lx=12;d=2017-05-13"
driver = webdriver.PhantomJS()
driver.get(url)
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.XPATH, "//div[@elt='results']//img[@class='LJTSM3-v-a']")))
print driver.current_url
driver.save_screenshot(r'flight_explorer.png')

result i got

Upvotes: 5

Related Questions