CODEWITHSUNDEEP
pythonpandasdataframeloopsappend
FlyUFalcon
FlyUFalcon

Reputation: 344

Pandas: import multiple dataframes, change headers and append them in loop

I was analysing crypto data using a library called ccxt. With ccxt, I can easily get candlestick for crypto datafeed into dataframe:

import pandas as pd
import numpy
import config
import ccxt
import json
import csv

exchange = ccxt.binance({
    'enableRateLimit': True, 
    'options': {'defaultType': 'future'}
})

 
name = exchange.fetch_ohlcv('BTC/USDT', limit=5, timeframe='1m')
btc = pd.DataFrame(name, columns=['timestamp', 'open', 'high', 'low', 'close', 'volume'])
 
print(btc)

single crypto Output:

       timestamp      open      high       low     close   volume
0  1620202740000  54945.31  54987.01  54945.30  54978.49  118.239
1  1620202800000  54978.49  55054.00  54972.04  55027.12  337.619
2  1620202860000  55027.12  55041.05  54950.05  54951.96  131.414
3  1620202920000  54951.96  55067.36  54951.95  55063.78  176.529
4  1620202980000  55063.79  55064.00  55000.00  55014.39  107.082

I wanted to import multiple cryptos into one dataframe with two-level headers, something like the following.

                      btc       btc        btc     btc     btc      eth        eth       eth      eth     eth                      
       timestamp     open      high       low     close   volume    open      high       low     close    volume  
0  1620202740000  54945.31  54987.01  54945.30  54978.49  118.239  54945.31  54987.01  54945.30  54978.49  4345
1  1620202800000  54978.49  55054.00  54972.04  55027.12  337.619  54945.31  54987.01  54945.30  54978.49  134.239   
2  1620202860000  55027.12  55041.05  54950.05  54951.96  131.414  54945.31  54987.01  54945.30  54978.49  14358.239
3  1620202920000  54951.96  55067.36  54951.95  55063.78  176.529  54945.31  54987.01  54945.30  54978.49  1148.239
4  1620202980000  55063.79  55064.00  55000.00  55014.39  107.082  54945.31  54987.01  54945.30  54978.49  18.239

In that way I can create columns for multiple crypto at the same time, such as

for x in ['btc', 'eth']:
    df.loc[:, (x, 'fast_ema_1min')] = df[x]['close'].rolling(window=1).mean()
    df.loc[:, (x, 'slow_ema_20min')] = df[x]['close'].rolling(window=20).mean()

I have tried

basket = [['BTC', 'BTC/USDT'], ['ETH', 'ETH/USDT'], ['ADA', 'ADA/USDT']]
appended_data = []
for i, name in basket:
    i = exchange.fetch_ohlcv(name, limit=5, timeframe='1m')
    i = pd.DataFrame(i, columns=[['_', name, name, name, name,name], ['timestamp', 'open', 'high', 'low', 'close', 'volume']])
    appended_data.append(i)
    print(appended_data)
print(appended_data.head(5))

However, I got error

AttributeError: 'list' object has no attribute 'head'

Can someone advise? Thanks.

Upvotes: 0

Views: 219

Answers (2)

FlyUFalcon
FlyUFalcon

Reputation: 344

Thanks to @SeaBean

I managed to solve the problem.

appended_data = []
 
basket = [['BTC', 'BTC/USDT'], ['ETH', 'ETH/USDT'], ['ADA', 'ADA/USDT']]
for i, name in basket:
    i = exchange.fetch_ohlcv(name, limit=5, timeframe='1m')
    i = pd.DataFrame(i, columns=[['_', name, name, name, name,name], ['timestamp', 'open', 'high', 'low', 'close', 'volume']])
    appended_data.append(i)
    df = pd.concat(appended_data, axis=1, ignore_index=True)

Upvotes: 0

SeaBean
SeaBean

Reputation: 23217

You can try retaining your code of aggregating a list of dataframes and then concat them after the loop, like below:

pd.concat(appended_data, axis=1, ignore_index=True)

Upvotes: 1

Related Questions