CODEWITHSUNDEEP
pythonbeautifulsouphtml-parsing
Afnan
Afnan

Reputation: 89

Get Structured Data from HTML using python and beautiful soup

I an new to python. I want to get the result of the code as below:

Score      Postive        Negative
  5         good            bad
  7       interesting
  3                       horrible

But my code output nothing.Please where is the problem?

from bs4 import BeautifulSoup
text = """
... <body>
        <div class="review">
        <p class="pos">good</p>
        <p class="neg">bad</p>
    </div>
    <div class="review">
        <p class="pos">interesting</p>
    </div>
    <div class="review">
        <p class="neg">horrible</p>
    </div>
... </body>"""
soup = BeautifulSoup(text)
for parent in soup.find_all('div', attrs={'class': 'review'}):   
if parent.findNextSiblings('p', attrs={'class': 'pos'}):
    postive.append(parent.get_text())
else:
    postive.append("")
if parent.findNextSiblings('p', attrs={'class': 'neg'}): 
    negtive.append(parent.get_text())
else:
    negtive.append("")

Upvotes: 1

Views: 290

Answers (1)

alecxe
alecxe

Reputation: 473873

p tags are not siblings of the div tag with class review, they are children:

positive = []
negative = []
for div in soup.find_all('div', attrs={'class': 'review'}):
    pos = div.find('p', {'class': 'pos'})
    positive.append(pos.get_text() if pos else '')

    neg = div.find('p', {'class': 'neg'})
    negative.append(neg.get_text() if neg else '')

print positive
print negative

Prints:

[u'good', u'interesting', ''] 
[u'bad', '', u'horrible']

Upvotes: 1

Related Questions