CODEWITHSUNDEEP
pythonhtmlweb-scrapingbeautifulsouphtml-parsing
learnerX
learnerX

Reputation: 1082

Python scraping (Beautiful Soup) to obtain data from this HTML

 <ul>
  <li>
    <div class="c_logo_box">
     <a href="money-transfer-companies/ria-money-transfer/"><img src="http://www.compareremit.com/uploads/ria-logo11.png" style="height:57px;width:147px;" alt="RIA Money Transfer"></a>
     <span class="rs"> <span class="txt13">&#8377;</span> 61.24</span>
       </div>
  </li>
 ...

I wish to scrap the name from 'alt = Ria Money Transfer' and rate from span 61.24.

So far I have this Python code:

#!/usr/bin/python

import requests
import re
from bs4 import BeautifulSoup

r = requests.get('http://www.compareremit.com')
data = r.text

soup = BeautifulSoup(data)
for rate in soup.find_all('li', re.compile('money')):
print rate.text

It gives me nothing. Could someone tell me what am I missing? Also, I'm having trouble visualizing which element I'm support to look for in the for loop search, could you clarify in general how to know what to specify as a condition in for loop in such cases?

Upvotes: 3

Views: 685

Answers (2)

Amal Ts
Amal Ts

Reputation: 861

Your code is logically not correct. You can do this in multiple ways, try this code

#!/usr/bin/python

import requests
import re
from bs4 import BeautifulSoup

r = requests.get('http://www.compareremit.com')
data = r.text

soup = BeautifulSoup(data)
for rate in soup.find_all('div',{"class":"c_logo_box"}):
    print rate.a.img['alt'] 
    print rate.span.text

Upvotes: 1

alecxe
alecxe

Reputation: 473873

There are multiple ways to reach the element. One option is to rely on the a tag, href of which contains the ria-money-transfer part, then get the following span element containing the rate:

import re

from bs4 import BeautifulSoup
import requests

response = requests.get('http://www.compareremit.com')
soup = BeautifulSoup(response.content)

link = soup.find('div', class_='c_logo_box').find('a', href=re.compile(r'ria-money-transfer'))
print(link.img.get('alt'))

rate = link.find_next_sibling('span').text.split(' ')[-1]
print(rate)

Prints:

RIA Money Transfer
61.24

Upvotes: 1

Related Questions