CODEWITHSUNDEEP
pythonseleniumweb-scraping
Amen Aziz
Amen Aziz

Reputation: 779

How to scrape data from website using selenium

I am try to scrape data but they will provide me an error this is website link https://www.dastelefonbuch.de/Suche/Zahnarzt

from selenium import webdriver
PATH="C:\Program Files (x86)\chromedriver.exe"
url='https://www.dastelefonbuch.de/Suche/Zahnarzt'
driver =webdriver.Chrome(PATH)
driver.get(url)
vid=driver.find_element_by_xpath("//div[@class='vcard']")
for item in vid:
    title=item.find_element_by_xpath("//div[@class='name']").text
    phone=item.find_element_by_xpath("//span[@class='nr']").text
    website=item.find_element_by_xpath("//i[@class='icon icon_url']").text
    print(title,phone,website)

Upvotes: 0

Views: 264

Answers (2)

Wonka
Wonka

Reputation: 1886

When you extract xpath on an item you have to change a bit xpath, saying lookup on current node with ".//"

Change

title=item.find_element_by_xpath("//div[@class='name']").text

to

title=item.find_element_by_xpath(".//div[@class='name']").text

Checkout and feedback if it works

Upvotes: 0

undetected Selenium
undetected Selenium

Reputation: 193058

Instead of vid=driver.find_element_by_xpath("//div[@class='vcard']") which returns a WebElement you need vid=driver.find_element_by_xpath("//div[@class='vcard']") which would return a list to iterate as follows:

from selenium.webdriver.common.by import By

vid = driver.find_elements(By.XPATH, "//div[@class='vcard']")
for item in vid:
    title=item.find_element_by_xpath("//div[@class='name']").text
    phone=item.find_element_by_xpath("//span[@class='nr']").text
    website=item.find_element_by_xpath("//i[@class='icon icon_url']").text
    print(title,phone,website)

Upvotes: 1

Related Questions