CODEWITHSUNDEEP
pythonseleniumparsingselenium-webdriver
Sinchetru
Sinchetru

Reputation: 571

How to extract the link from selenium webdriver element?

I did earlier a parser for a site and now I want to do the same thing for the other https://999.md/ru/list/real-estate/apartments-and-rooms?view_type=photo but I have a problem. I want to extract the link from the following code:

<ul class="ads-list-photo">   
    <li class="ads-list-photo-item"> 
       <div class="ads-list-photo-item-thumb">
         <a href="/ru/39854705">

I am trying to do that using selenium webdriver and the following code:

driver.get(url)
driver.find_element_by_xpath('//*[@id="container"]/div/section/nav/ul/li[2]/a').click() # acces Imobiliare
driver.find_element_by_link_text("Apartamente şi camere").click()

parentElement = WebDriverWait(driver, 10)
parentElement = parentElement.until(EC.visibility_of_element_located((By.XPATH, '//*[@id="js-pjax-container"]')))
elementList = parentElement.find_elements_by_tag_name("li") #Acces lista cu obiecte

links = []

for element in driver.find_elements_by_xpath('//*[@id="js-ads-container"]/ul/li[1]/div[1]'):
    links.append(element.get_attribute('href'))
links

but instead of obtaining alit of links I get a list of None values. How to solve that?

Upvotes: 2

Views: 1419

Answers (1)

Pedro Lobito
Pedro Lobito

Reputation: 98921

I guess you can use:

driver = webdriver.Firefox()
driver.get("https://999.md/ru/list/real-estate/apartments-and-rooms?view_type=photo")
links = driver.find_elements_by_xpath("//div[@class='ads-list-photo-item-thumb']//a")

for x in links:
    print x.get_attribute('href')

Output:

https://999.md/ru/40057219
https://999.md/ru/22587357
https://999.md/ru/38650049
https://999.md/ru/17528096
https://999.md/ru/31246607
https://999.md/ru/12459330
https://999.md/ru/8449720
https://999.md/ru/34011321
...

Upvotes: 3

Related Questions