CODEWITHSUNDEEP
pythonseleniumloopselement
Myk
Myk

Reputation: 279

How to iterate over children webelements in Python Webbot/Selenium?

I have a table of search results in Selenium browser and each search result is defined in Html like this:

<div class="item
            itemWrapper
            ItemPosition1
            ItemMonitor
            " data-position="1" data-it-name="NAME OF THE ITEM" data-it-category="Category" role="article">

    <div class="item-image">

        <a href="/some/link/" target="_blank" rel="noopener" class="itemRec">
            <img src="https://some.jpg" alt="some name" class="img-responsive">
        </a>
                        
    </div>

    <h2 class="small-text item-title">
        <a href="/some/other/link" class="sendItem" target="_blank" rel="noopener">Link Text</a>
    </h2>

    <div class="item-bottom">

        <div class="pull-left item-price">
                <span>999</span>
        </div>

        <div class="pull-right detail-link">
            <a href="/link/to/detail" title="link title" class="detail"
                Detail
            </a>        
        </div>
    </div>
</div>

I am able to find all webelements by classname = item.

elements = driver.find_elements_by_class_name("item")

I would need to iterate over elements and get their position, name and price to be able to click to one of them:

for e in elements:
   position=e.get_attribute("data-position").value, 
   name=e.get_attribute("data-it-name").value, 
   price=e.find_element(By.CLASS_NAME,'item-price').value

but this does not work - get_attribute returns None and find_element does not find any child element

Can you please advise me how to get the "data-" atributes and child elements values correctly?

Whole code using Webbot:

import webbot
from selenium.webdriver.common.by import By
web = webbot.Browser()

web.go_to('www.***.cz')
web.type('bed', classname='header-search-form')
web.press(web.Key.ENTER)
elements = web.find_elements(classname="product-item")
for e in elements:
    name = e.get_attribute("data-it-name").value
    price = e.find_element(By.CLASS_NAME, 'item-price').value
    print(name,price)
    break

Upvotes: 0

Views: 459

Answers (1)

alecxe
alecxe

Reputation: 473813

classname acts weirdly in webbot. You definitely are not getting a product item there:

In [56]: elements[0].get_attribute('outerHTML')
Out[56]: '<a href="/">\n\n\t\t\t\t\t\t<img src="https://s.favi.cz/static/frontend/_global/images/favi-logo/favi-logo.60d511aff13247dd52f15acf6bdf2af9.svg" role="banner">\n\n\t\t\t\t\t</a>'

Works well with a CSS selector:

In [58]: elements = web.find_elements(css_selector=".product-item")

In [59]: elements[0].get_attribute('outerHTML')
Out[59]: '<div class="\n\t\t\tproduct-item\n\t\t\titemWrapper\n\t\t\tproductItemPosition1\n\t\t\tproductItemMonitor\n\t\t\tproductItemWrapper\n\t\t\tsendProductTransactionWrapper\n\t\t\t\t\t" data-position="1" data-pr-name="Moderní box spring postel Alvares 160x200, bílá" data-tr-id="04d62b60-9d00-4d1b-b03c-2258c50bfdb9" data-pr-category="Postele" data-tr-ob-id="2144583" data-m-ob-id="2345478" role="article">\n\n\t\t<div class="product-image">\n\n\t\t\t<a href="/produkty/nakupovat/2345478/1/84/1/searchBlockSearch/04d6ec5f-0aaf-458e-8754-58f7ccf4e971/04d62b60-9d00-4d1b-b03c-2258c50bfdb9" target="_blank" rel="noopener" class="\n\t\t\t\t\tsendProductTransaction\n\t\t\t\t\tproductRecommendation\t\t\t\t\t\t\t\t\t">\n\t\t\t\t\t\t\t\t\t<img src="https://s.favi.cz/static/images/t/product/300/6f/92/6f922779-bc84-483e-b1cd-ad8522ef0c92.jpg" alt="Moderní box spring postel Alvares 160x200, bílá" class="img-responsive">\n\t\t\t\t\t\t\t</a>\n\n\t\t\t<a href="#" title="Líbí se mi" class="\n\t\t\t\t\tuser-like\n\t\t\t\t\tuserLike\n\t\t\t\t\tproductRecommendation\t\t\t\t\t\t\t\t\t\twith-count\t\t\t\t" data-like-product-master-id="2144583">\n\t\t\t\t\t\t\t\t\t<span class="count">485</span>\n\t\t\t\t\t\t\t</a>\n\n\t\t\t\n\t\t\t\n\t\t</div>\n\n\t\t<div class="product-labels stickers-holder">\n\n\t\t\t\t\t\t\t<span class="sticker storage white">\n\t\t\t\t\t<span class="text">Skladem</span>\n\t\t\t\t</span>\n\t\t\t\n\t\t\t\n\t\t\t\n\t\t\t\n\t\t</div>\n\n\t\t<h2 class="small-text product-item-title">\n\t\t\t<a href="/produkty/nakupovat/2345478/1/84/1/searchBlockSearch/04d6ec5f-0aaf-458e-8754-58f7ccf4e971/04d62b60-9d00-4d1b-b03c-2258c50bfdb9" class="\n\t\t\t\t\tsendProductTransaction\n\t\t\t\t\tproductRecommendation\t\t\t\t" target="_blank" rel="noopener">Moderní box spring postel Alvares 160x200, bílá</a>\n\t\t</h2>\n\n\t\t<div class="product-bottom">\n\n\t\t\t<div class="pull-left product-item-price">\n\t\t\t\t<span>15 599 Kč</span>\n\t\t\t\t\t\t\t</div>\n\n\t\t\t<div class="pull-right product-shop-link">\n\t\t\t\t<a href="/produkty/p/luxusni-box-spring-postel-alves-160x200-bila?v=2345478" title="Moderní box spring postel Alvares 160x200, bílá" class="btn btn-bordered btn-sm sendProductTransaction">\n\t\t\t\t\tDetail\n\t\t\t\t</a>\n\n\t\t\t\t<a href="/produkty/nakupovat/2345478/1/84/1/searchBlockSearch/04d6ec5f-0aaf-458e-8754-58f7ccf4e971/04d62b60-9d00-4d1b-b03c-2258c50bfdb9" target="_blank" rel="noopener" class="\n\t\t\t\t\t\tbtn\n\t\t\t\t\t\tbtn-std\n\t\t\t\t\t\tbtn-sm\n\t\t\t\t\t\tproductRecommendation\t\t\t\t\t\tsendProductTransaction\n\t\t\t\t\t">\n\t\t\t\t\t<strong>Do obchodu</strong>\n\t\t\t\t</a>\n\t\t\t</div>\n\n\t\t</div>\n\n\t\t\n\t</div>'

In [60]: elements[0].get_attribute('data-position')
Out[60]: '1'

In [61]: elements[0].get_attribute('data-pr-name')
Out[61]: 'Moderní box spring postel Alvares 160x200, bílá'

Upvotes: 1

Related Questions