CODEWITHSUNDEEP
pythonhtmlpython-2.7beautifulsouphtml-parsing
swateek
swateek

Reputation: 7580

Python: Extract all childs of a <div> tag using BeautifulSoup

The tags are like this:

<div class="zg_itemWrapper" style="height:315px">
<div class="zg_title"><a href="http://www.amazon.in/Taste-Suspense-Action-Thriller-Mystery-ebook/dp/B00JKN41ZS/ref=zg_bs_1637004031_f_2">The Taste of Fear (A Suspense Action...</a>
</div>
<div class="zg_byline">by Jeremy Bates</div>
<div class="zg_price">Free</div>
</div>

<div class="zg_itemWrapper" style="height:315px">
<div class="zg_title"><a href="http://www.amazon.in/Taste-Suspense-Action-Thriller-Mystery-ebook/dp/B00JKN41ZS/ref=zg_bs_1637004031_f_2">Another Book</a>
</div>
<div class="zg_byline">by Jeremy</div>
<div class="zg_price">Free</div>
</div>

I am using BeautifulSoup to read the webpage and extract a few details:

Title, Author, Price and Link

The code that I have tried could extract only one of them, but I want all of it in a collection per title.

items = soup.find_all("div", {"class":"zg_itemWrapper"})

for item in items:
    titles = item.find_all("div", {"class":"zg_title"})
    for title in titles:
        print title.text

Upvotes: 1

Views: 1514

Answers (1)

alecxe
alecxe

Reputation: 474151

You are on the right track.

Use find by class name for every "itemWrapper" found:

items = soup.find_all("div", {"class":"zg_itemWrapper"})

for item in items:
    title_elm = item.find("div", {"class":"zg_title"}).a
    title = title_elm.get_text()
    link = title_elm["href"]

    author = item.find("div", {"class": "zg_byline"}).get_text()
    price = item.find("div", {"class": "zg_price"}).get_text()

    print title, link, author, price

Upvotes: 1

Related Questions