xpath selector returns empty values

Question

i'm trying to manipulate text from all spans under "critical-product-marquee-container" div using python, selenium and xpath selector.


    
        
            
                Surgical & Reusable
                            Masks: 1,215,028,138 units Surface Wipes: 56,023,303 units Gowns and Coveralls: 22,899,306 units Respirators: 11,732,566 units Face Shields: 4,731,581 units Hand Sanitizer: 527,672 L Surface Solutions: 48,337 L Ventilators: 500 units Thermometers: 6,922 units Testing Kits: 22,205 units Disposable Gloves: 137,455,881 units 
            
            
                Surgical & Reusable
                            Masks: 1,215,028,138 units Surface Wipes: 56,023,303 units Gowns and Coveralls: 22,899,306 units Respirators: 11,732,566 units Face Shields: 4,731,581 units Hand Sanitizer: 527,672 L Surface Solutions: 48,337 L Ventilators: 500 units Thermometers: 6,922 units Testing Kits: 22,205 units Disposable Gloves: 137,455,881 units

my code is as follows:

from selenium import webdriver 
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = webdriver.ChromeOptions();
options.add_argument('headless');
driver = webdriver.Chrome(options=options)
url = 'https://www.rrpcanada.org/#/' # site I'm scraping
driver.get(url)

item_elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, '//div[@class="critical-product-marquee-container"]//span[@class="bold"]')))
quantity_elements = WebDriverWait(driver, 20).until(EC.presence_of_all_elements_located((By.XPATH, '//div[@class="critical-product-marquee-container"]//span[contains(@class, "line-item-bold")]')))

it returns a lost of all span elements but when trying to print the span text it prints the first two spans only, the rest are empty. any advice?

cruisepandey · Accepted Answer

basically that's marquee in HTML5, so you have to explicitly wait for each elements.

Code :

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(50)
driver.get("https://www.rrpcanada.org/#/")
wait = WebDriverWait(driver, 50)
try:
    wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[class$='unset-margin']"))).click()
except:
    pass

elements = driver.find_elements(By.XPATH, "//div[contains(@class, 'marquee-text-text')]//descendant::span[@class='bold']")
for ele in elements:
    wait.until(EC.visibility_of(ele))
    print(ele.text)
    new_ele = ele.find_element_by_xpath(".//following-sibling::span")
    wait.until(EC.visibility_of(new_ele))
    print(new_ele.text)

Output :

"C:\Program Files\Python39\python.exe" C:/Users/****/PycharmProjects/SeleniumSO/Chrome.py
Surgical & Reusable Masks:
1,215,028,138 units
Surface Wipes:
56,023,303 units
Gowns and Coveralls:
22,899,306 units
Respirators:
11,732,566 units
Face Shields:
4,731,581 units
Hand Sanitizer:
527,672 L
Surface Solutions:
48,337 L
Ventilators:
500 units
Thermometers:
6,922 units
Testing Kits:
22,205 units
Disposable Gloves:
137,455,881 units
Surgical & Reusable Masks:
1,215,028,138 units
Surface Wipes:
56,023,303 units
Gowns and Coveralls:
22,899,306 units
Respirators:

xpath selector returns empty values

Answers (1)

Related Questions