CODEWITHSUNDEEP
pythonbeautifulsoup
fbelfort
fbelfort

Reputation: 293

Problems extracting text from multiple tags ignoring subtags

I have this sample-html:

soup=BeautifulSoup('''<ul>
 <li class=“item">
 <span class="letter">A. </span>
Text I want </li>
 <li class="item">
 <span class="letter">B.</span>                           
Second text I want</li></ul>''')

I'm trying to extract "Text I want" and "Second text I want", ignoring the span tags. So far what I have done:

soup.li.find_all(text=True,recursive=False)

Which returns ['\n', '\nText I want '].

If I try:

for s in soup.ul:
    print(s.find(text=True,recursive=False))

I get an error:

TypeError: find() takes no keyword arguments
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-93-f253cd430e83> in <module>
      1 for s in soup.ul:
----> 2     print(s.find(text=True,recursive=False))

TypeError: find() takes no keyword arguments

Any help is appreciated.

Upvotes: 1

Views: 36

Answers (1)

Andrej Kesely
Andrej Kesely

Reputation: 195408

You can use list-comprehension to extract the texts:

from bs4 import BeautifulSoup

soup = BeautifulSoup(
    """<ul>
 <li class="item">
 <span class="letter">A. </span>
Text I want </li>
 <li class="item">
 <span class="letter">B.</span>                           
Second text I want</li>""",
    "html.parser",
)

texts = [
    txt
    for li in soup.select("li.item")
    for t in li.find_all(text=True, recursive=False)
    if (txt := t.strip())
]
print(texts)

Prints:

['Text I want', 'Second text I want']

Or remove the <span> first and then get text:

for span in soup.select("span"):
    span.extract()

texts = [li.get_text(strip=True) for li in soup.select("li.item")]
print(texts)

Prints:

['Text I want', 'Second text I want']

Or: Find <span> and then .find_next_sibling(text=True):

texts = [
    li.find_next_sibling(text=True).strip()
    for li in soup.select("li.item span")
]
print(texts)

Prints:

['Text I want', 'Second text I want']

Upvotes: 1

Related Questions