CODEWITHSUNDEEP
pythonseleniumweb-scraping
Omega
Omega

Reputation: 871

Trouble fetching text from div tag using Selenium

I'm trying to scrape the tables which sit under the "Fuel and Service Rates" header on this site: https://www.signatureflight.com/locations/acy. This includes the "$5.83 100LL Full Service" bit as well as the table below with fees and additional benefits.

I can locate the first part by using driver.find_element_by_xpath('//span[contains(@data-bind, "text: formatCurrency")]').text, but this doesn't print anything. I'm using 'contains' in case the fuel type differs for other airports.

Any help would be appreciated.

HTML for reference:

<div class="col-md-8 col-xs-7 fuelWrapper"> 
<span style="color: #00263d; font-size:25px" 
data-bind="text: formatCurrency(Fuel100LL)">$5.83</span><br> 
<span style="color: #00263d; font-size: 11px">100LL Full Service</span> </div>

Upvotes: 0

Views: 39

Answers (1)

Thaer A
Thaer A

Reputation: 2323

The element ('//span[contains(@data-bind, "text: formatCurrency")]') can't be scraped because it's in a collapsed accordion.

The best solution I could come up with is to scroll to an element below the "Fuel and Service Rates" button and then click on it and then scrape the price.

I'm sure someone else can come up with a better solution with fewer lines of code.

Explanation in code comment.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Chrome('./chromedriver')

driver.get("https://www.signatureflight.com/locations/acy")
# Get an element below the fees section to scroll to, so the Fuel & Service Rate will be visible
scroll_to = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, "skyvector-heading")))
actions = ActionChains(driver)
# Scroll to that element
actions.move_to_element(scroll_to).perform()
# Find the Fuel & Service Rate link and click on it
btn = driver.find_element_by_css_selector("a[href='#fees']")
btn.click()
# Get the price
price = driver.find_element_by_xpath('//span[contains(@data-bind, "text: formatCurrency")]').text
print(price)

Output:

$5.83

Upvotes: 2

Related Questions