CODEWITHSUNDEEP
pythonseleniumweb-scrapingxpath
Amen Aziz
Amen Aziz

Reputation: 779

XPath issue using Selenium

I am trying to scrape title, but they say your XPath expression is wrong

from selenium.webdriver.common.by import By
from selenium import webdriver
from time import sleep

PATH = "C:\Program Files (x86)\chromedriver.exe"
url = 'https://www.nationalhardwareshow.com/en-us/attend/exhibitor-list.html'
driver = webdriver.Chrome(PATH)
driver.get(url)
sleep(2)

def searchplace():
    vid = driver.find_elements(By.XPATH, "//div[@class='row']")
    for item in vid:
        title = item.find_element_by_xpath(".//div[@class='company-info']//h3").text
        print(title)

searchplace()

Upvotes: 2

Views: 204

Answers (2)

Prophet
Prophet

Reputation: 33361

You are using the wrong locator here.

Match each vid block with this XPath expression: //div[contains(@class,'directory-item directory-item-feature-toggled')].

With that, your code will look like:

from selenium.webdriver.common.by import By
from selenium import webdriver
from time import sleep

PATH = "C:\Program Files (x86)\chromedriver.exe"
url = 'https://www.nationalhardwareshow.com/en-us/attend/exhibitor-list.html'
driver = webdriver.Chrome(PATH)
driver.get(url)
sleep(2)

def searchplace():
    vid = driver.find_elements(By.XPATH, "//div[contains(@class,'directory-item directory-item-feature-toggled')]")
    for item in vid:
        title = item.find_element_by_xpath(".//div[@class='company-info']//h3").text
        print(title)

searchplace()

I would advice you using Expected Conditions explicit waits instead of hardcoded pauses. With it, your code will be:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver
from time import sleep

PATH = "C:\Program Files (x86)\chromedriver.exe"
url = 'https://www.nationalhardwareshow.com/en-us/attend/exhibitor-list.html'
driver = webdriver.Chrome(PATH)
wait = WebDriverWait(driver, 20)

driver.get(url)
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[contains(@class,'directory-item directory-item-feature-toggled')]")))

sleep(0.3) # Leave a short delay to make sure not only the first item got visible

def searchplace():
    vid = driver.find_elements(By.XPATH, "//div[contains(@class,'directory-item directory-item-feature-toggled')]")
    for item in vid:
        title = item.find_element_by_xpath(".//div[@class='company-info']//h3").text
        print(title)

searchplace()

Upvotes: 1

Vignesh
Vignesh

Reputation: 231

The below XPath expression worked for me. Give it a try. Also try to reduce the XPath length efficiently if you need.

vid = driver.find_elements(By.XPATH, "//div[@class='directory-item directory-item-feature-toggled exhibitor-category']")
for item in vid:
    title = item.find_element(By.XPATH, "div[@class='row']/div[2]/div//div[@class='company-info']/div/a/h3")
    print(title.text)

Upvotes: 0

Related Questions