CODEWITHSUNDEEP
pythonpython-2.7
Napoleon
Napoleon

Reputation: 909

python scraping exchange rate data

An application that a friend uses depends on daily exchange rate figures from a particular site link to source of rates

The problem is there is no set time when the rate is changed which is affecting business since sometimes when the rate is changed she might be out and so until he comes back any transaction that happens will use the last rate entered. Sometimes she wins other times she looses out. I'm trying to create an automated client that will scrape and update the exchange rate for her independently.

So far I have been able to strip the content of the site down to a list:

[
<td style="text-align: left;">U.S Dollar</td>,
<td>USDGHS</td>, <td>1.8673</td>, <td>1.8994</td>,
<td style="text-align: left;">Pound Sterling</td>,
<td>GBPGHS</td>, <td>3.0081</td>, <td>3.0599</td>,
<td style="text-align: left;">Swiss Franc</td>,
<td>CHFGHS</td>, <td>2.0034</td>, <td>2.0375</td>,
<td style="text-align: left;">Australian Dollar</td>,
<td>AUDGHS</td>, <td>1.9667</td>, <td>2.0009</td>,
<td style="text-align: left;">Canadian Dollar</td>,
<td>CADGHS</td>, <td>1.8936</td>, <td>1.9259</td>,
<td style="text-align: left;">Danish Kroner</td>,
<td>DKKGHS</td>, <td>0.3255</td>, <td>0.3311</td>,
<td style="text-align: left;">Japanese Yen</td>,
<td>JPYGHS</td>, <td>0.0226</td>, <td>0.0230</td>,
<td style="text-align: left;">New Zealand Dollar</td>,
<td>NZDGHS</td>, <td>1.5690</td>, <td>1.5964</td>,
<td style="text-align: left;">Norwegian Kroner</td>,
<td>NOKGHS</td>, <td>0.3307</td>, <td>0.3363</td>]

I'm now strugling a bit to create a dictionary like so

{USDGHS: [1.8673, 1.8994], GBPGHS: [3.0081, 3.0599], ...}

I'll then use the dictionary to update the appropriate table in the database.

I got to this stage by using beautifulsoup4 and urllib2

[Edit]

Code that got me to this point

from bs4 import BeautifulSoup
import urllib2

url = "http://bog.gov.gh/data/bankindrate.php"
page = urllib2.urlopen(url)

soup = BeautifulSoup(page.read())

td = soup.find_all('td')

another_soup = BeautifulSoup(td[:-3])

print another_soup

Upvotes: 1

Views: 2715

Answers (1)

Jon Clements
Jon Clements

Reputation: 142206

You need to first find the rows (tr tags) and use those to then get the columns (td tags):

currencies = {}
trs = soup.find_all('tr') # find rows
for tr in trs[1:-3]: # skip first and last 3 (or whatever)
    text = list(tr.strings) # content of all text stuff in tr (works in this case)
    # [u'U.S Dollar', u'USDGHS', u'1.8673', u'1.8994']
    currencies[text[1]] = [float(text[2]), float(text[3])]

And put those into a dictionary using the appropriate key with a value of the two numbers converted to floats...

>>> currencies
{u'USDGHS': [1.8673, 1.8994], u'JPYGHS': [0.0226, 0.023], u'CHFGHS': [2.0034, 2.0375], u'CADGHS': [1.8936, 1.9259], ...}

Upvotes: 3

Related Questions