CODEWITHSUNDEEP
pythonpython-3.xweb-scrapingbeautifulsouppython-requests
Janko Pjer
Janko Pjer

Reputation: 37

How can I get the next child in BeautifulSoup

This is the HTML and code I have:

<a class="card__article-link" href="linktoarticle" title="articletitle">
<span class="card__egida">TEXT</span>
<span class="card__title ">TITLE</span>
<span class="card__subtitle">SUBTITLE</span>
</a>

import requests
from bs4 import BeautifulSoup
r = requests.get("link").text
soup = BeautifulSoup(r, "html.parser")

for span in soup.find_all("span", {"class": "card__egida"}):
    print(span.get_text())

The code correctly prints TEXT but I want the code to also print TITLE and SUBTITLE. I have tried with nextSibling but with no success. How can I do that?

Upvotes: 3

Views: 199

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195418

You can use .find_next() to get next elements:

from bs4 import BeautifulSoup


txt = '''<a class="card__article-link" href="linktoarticle" title="articletitle">
<span class="card__egida">TEXT</span>
<span class="card__title ">TITLE</span>
<span class="card__subtitle">SUBTITLE</span>
</a>'''

soup = BeautifulSoup(txt, 'html.parser')    

for span in soup.find_all("span", {"class": "card__egida"}):
    egida = span.get_text()
    title = span.find_next(class_='card__title').get_text()
    subtitle = span.find_next(class_='card__subtitle').get_text()

    print(egida)
    print(title)
    print(subtitle)

Prints:

TEXT
TITLE
SUBTITLE

Or: you can select the parent <a> and then search for title, subtitle, etc...:

for a in soup.select('a.card__article-link'):
    egida = a.select_one('.card__egida').get_text()
    title = a.select_one('.card__title').get_text()
    subtitle = a.select_one('.card__subtitle').get_text()

Upvotes: 3

Related Questions