CODEWITHSUNDEEP
pythonbeautifulsoup
wasmachien
wasmachien

Reputation: 1009

BeautifulSoup - getting rid of paragraph whitespace/line breaks

similarlist = res.find_all_next("div", class_="result-wrapper")
for item in similarlist:
    print(item)

This returns:

<div class="result-wrapper">
<div class="row-fluid result-row">
<div class="span6 result-left">
<p>
<a class="tooltipLink warn-cs" data-original-title="Listen" href="..." rel="tooltip"><i class="..."></i></a>
<a class="muted-link" href="/dictionary/german-english/aa-machen">Aa <b>machen</b></a>
</p>
</div>   
<div class="span6 result-right row-fluid">
<span class="span9">
<a class="muted-link" href="/dictionary/english-german/do-a-poo">to do a poo</a>, <a class="muted-link" href="/dictionary/english-german/pooh">to pooh</a>
</span>
</div>
</div>
</div>

When I choose to print item.get_text() instead, I get

abgeneigt machen
to disincline




abhängig machen
2137

to predicate




Absenker machen
to layer

So basically a lot of new lines between the list items that I don't need. Is this because of the <p> tags? How do I get rid of them?

Upvotes: 3

Views: 6019

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1122232

Yes, between tags the HTML contains whitespace (including newlines) too.

You can easily collapse all multi-line whitespace with a regular expression:

import re

re.sub(r'\n\s*\n', r'\n\n', item.get_text().strip(), flags=re.M)

This removes any whitespace (newlines, spaces, tabs, etc.) between two newlines.

Upvotes: 7

ueg1990
ueg1990

Reputation: 1033

You can the the strip() function in python

item.get_text().strip()

Upvotes: -3

Related Questions