CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoupscreen-scraping
Mithun Manohar
Mithun Manohar

Reputation: 586

python data scraping change currency

I am trying to fetch bitcoin and ethereum historical price from coinmarketcap.com using the below script from page https://coinmarketcap.com/currencies/bitcoin/historical-data/?start=20130428&end=20180216:

import requests
import xlwt
import traceback
import pandas as pd
import urllib.request as req
from bs4 import BeautifulSoup
from pandas import ExcelWriter

def process_data(data, coin, workbook):
  try:
    sheet = workbook.add_sheet(coin)
    soup = BeautifulSoup(data, 'lxml')
    table = soup.find_all('table')[0]

    df = pd.read_html(str(table))
    k = (df[0].to_json(orient='records'))
    import json
    resp = json.loads(k)
    # resp format: {'Date': 'Apr 28, 2013', 'Open': 135.3, 'High': 135.98, 'Low': 132.1, 'Close': 134.21, 'Volume': '-', 'Market Cap': 1500520000}
    lst = [[each['Date'], each['Open'], each['High'], each['Low'], each['Close'], each['Volume'], each['Market Cap']] for each in resp]

    for i, l in enumerate(lst):
      for j, col in enumerate(l):
        sheet.write(i, j, col)
  except Exception as e:
    print (e)
    print(traceback.print_exc())

coins = ['Bitcoin', 'Ethereum']

workbook = xlwt.Workbook(encoding='ascii')

if __name__ == '__main__':
    for each in coins:
      coin = each.lower()
      url = "https://coinmarketcap.com/currencies/"+ coin + "/historical-data/?start=20090428&end=20180207"
      print (url)
      try:
        a = requests.get(url)
        process_data(a.text, each, workbook)
      except Exception as e:
        print 'error in fetching data', coin
    workbook.save('cmc_data_f.xls')

The script gets the html response from the page and write to an excel file. The problem is the webpage is returning data in USD by default. I want the data in EURO.

On the website there is a drop down to choose from multiple currencies. But when I send the request from python it is returning USD by default.

Does anyone know if there a way to request webpage in EURO currency from coinmarketcap.com?

Upvotes: 1

Views: 1377

Answers (1)

Samuel Henderson
Samuel Henderson

Reputation: 141

No, but you can pull the data you need from the page, to match the conversion the page is doing.

After the table = ... bit, add this line to get the exchange rate:

usd_per_eur = float(soup.find("div",{"id":"currency-exchange-rates"})['data-eur'])

And then tweak your loop code, something like this:

for i, l in enumerate(lst):
  for j, col in enumerate(l):
    if 0 < j < 5: # indices 1-4 of response contain USD values
        try:
            converted = float(col)/usd_per_eur
        except ValueError:
            pass
        else:
            col = str(round(converted,2))
    sheet.write(i, j, col)

Upvotes: 1

Related Questions