CODEWITHSUNDEEP
pythonbeautifulsoup
Peter
Peter

Reputation: 593

Extract string from tag with BeautifulSoup

I am trying to extract from below table. I cut it after the second <td>, with six more to follow. All eight strings need to be extracted, e.g. in the example below I would want the values 61.5, 56.43, etc.

The code below only gives me the first value, 61.5. How can I get the remaining values? 

 soup.find("div", {"class":"value"}).text


<td class="flow">
    <div class="heading" style="min-height: 63px;">Dornum</div>
    <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
</td>
<td class="flow">
    <div class="heading" style="min-height: 63px;">Emden EMS</div>
    <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
</td>

Upvotes: 15

Views: 34294

Answers (3)

geekpradd
geekpradd

Reputation: 418

You will need to get all the values using find_all() in BeautifulSoup.

from bs4 import BeautifulSoup

html = '''<td class="flow">
    <div class="heading" style="min-height: 63px;">Dornum</div>
    <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
</td>
<td class="flow">
    <div class="heading" style="min-height: 63px;">Emden EMS</div>
    <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
</td>'''

soup = BeautifulSoup(html)
out=[]
for elem in soup.find_all("div", {"class":"value"}):
    out.append(elem.text)

Upvotes: 0

pacifier
pacifier

Reputation: 181

 <td class="flow">
 <div class="heading" style="min-height: 63px;">Dornum</div>
 <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
 </td>
 <td class="flow">
 <div class="heading" style="min-height: 63px;">Emden EMS</div>
 <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
 </td>

try this out :

 temp = soup.select('div[class="value"]') 
 result = []
 for i in temp:
     result.append(i.get_text())

Upvotes: 3

mhawke
mhawke

Reputation: 87054

Use soup.find_all() to obtain a list of matching elements, then grab the text attribute for each element:

from bs4 import BeautifulSoup

html = '''<td class="flow">
    <div class="heading" style="min-height: 63px;">Dornum</div>
    <div class="data"><div class="value">61.5</div> MSm<sup>3</sup>/d</div>
</td>
<td class="flow">
    <div class="heading" style="min-height: 63px;">Emden EMS</div>
    <div class="data"><div class="value">56.43</div> MSm<sup>3</sup>/d</div>
</td>'''

soup = BeautifulSoup(html)

data = [element.text for element in soup.find_all("div", "value")]

>>> data
[u'61.5', u'56.43']

Or, if you want them as floats:

data = [float(element.text) for element in soup.find_all("div", "value")]
>>> data
[61.5, 56.43]

Upvotes: 12

Related Questions