CODEWITHSUNDEEP
pythonbeautifulsouppython-requestspicklepandas-datareader
realnickspiess
realnickspiess

Reputation: 47

Importing Data from CoinMarketCap

I am following a Youtube tutorial that is importing data from Yahoo for the S&P500 and I am attempting to instead import data from coinmarketcap to track cryptocurrency data.

I am attempting to use Pandas DataReader to get data from coinmarketcap but am getting an error (NotImplementedError: data_source='coinmarketcap' is not implemented).

I am not sure if my syntax is wrong or if I'm missing a module or I am using the incorrect data_source implementation.

Here's the code:

import coinmarketcap

import bs4 as bs
import datetime as dt
import os
import pandas_datareader as web
import pickle
import requests

def save_cmc_tickers():
    resp = requests.get('https://coinmarketcap.com')
    soup = bs.BeautifulSoup(resp.text, "lxml")
    table = soup.find('table', {'class': 'table floating-header'})
    #table1 = soup.find('table', {'class': 'no-wrap text-right'})
    tickers = []
    for row in table.findAll('tr')[1:]:
        ticker = row.findAll('a')[0].text
        price = row.findAll('a', {'class': 'price'})[0].text
        tickers.append(ticker)
        tickers.append(price)

    with open("cmctickers.pickle", "wb") as f:
        pickle.dump(tickers, f)

    print(tickers)

    return tickers

    #save_cmc_tickers()

    def get_data_from_cmc(reload_cmc100=False):

    if reload_cmc100:
        tickers = save_cmc_tickers()
    else:
        with open("cmctickers.pickle", "rb") as f:
            tickers = pickle.load(f)

    if not os.path.exists('stock_dfs'):
        os.makedirs('stock_dfs')

    start = dt.datetime(2010, 1, 1)
    end = dt.datetime(2017, 12, 31)

    for ticker in tickers:
        print(ticker)
        if not os.path.exists('stock_dfs/{}.csv'.format(ticker)):
            df = web.DataReader(ticker, 'coinmarketcap', start, end)
            df.to_csv('stock_dfs/{}.csv'.format(ticker))
        else:
            print("Already have {}".format(ticker))

get_data_from_cmc()

I was also getting a "EOFError: Ran out of input" error earlier, so I don't know if that has anything to do with it, but that is no longer appearing.

Any information would help greatly. Thanks.

Upvotes: 0

Views: 1896

Answers (1)

Addison Lynch
Addison Lynch

Reputation: 685

pandas_datareader does not provide support for CoinMarketCap. This is why a NotImplementedError has been raised.

Separate note: It may not be necessary to scrape the CoinMarketCap for the data you are seeking, as CoinMarketCap has a developer API (link) which could be accessed as follows (after obtaining an API key), though only certain endpoints are covered in the free plan.

# python 3
import requests

url = "https://pro-api.coinmarketcap.com/v1/cryptocurrency/ohlcv/historical"

params = {
    "time_start": "2017-01-01",
    "time_end": "2018-01-01",
    "interval": "1d"
}

headers = {
    "X-CMC_PRO_API_KEY": "<YOUR_API_KEY>"   
}

r = requests.get(url, params=params, headers=headers)
r.json()

Upvotes: 1

Related Questions