CODEWITHSUNDEEP
pandasdataframeibpy
Panda_User
Panda_User

Reputation: 199

convert IbPy data request into pandas dataframe

I have the below code, which downloads historical data using IbPy from Interactive Brokers, and saves this to csv. Instead of saving it to csv, I would like to directly feed the data into a pandas dataframe (bypassing the csv bit).

Can anyone help?

from time import sleep, strftime, localtime  
from ib.ext.Contract import Contract  
from ib.opt import ibConnection, message  


new_symbolinput = ['ES', 'NQ']
newDataList = []  
dataDownload = []  

def historical_data_handler(msg):  
    global newDataList  
    print (msg.reqId, msg.date, msg.close)
    if ('finished' in str(msg.date)) == False:  
        new_symbol = new_symbolinput[msg.reqId]  
        dataStr = '%s, %s, %s' % (new_symbol, strftime("%Y-%m-%d", localtime(int(msg.date))), msg.close)  
        newDataList = newDataList + [dataStr]
    else:  
        new_symbol = new_symbolinput[msg.reqId]  
        filename = 'minutetrades' + new_symbol + '.csv'  
        csvfile = open('csv_day_test/' + filename,'w')  
        for item in newDataList:  
            csvfile.write('{} \n'.format(item))
        csvfile.close()  
        newDataList = []  
        global dataDownload  
        dataDownload.append(new_symbol)  

con = ibConnection()  
con.register(historical_data_handler, message.historicalData)  
con.connect()  

symbol_id = 0  
for i in new_symbolinput:  
    print (i)  
    qqq = Contract()  
    qqq.m_symbol = i  
    qqq.m_secType = 'FUT'  
    qqq.m_exchange = 'GLOBEX'  
    qqq.m_currency = 'USD'
    qqq.m_expiry = '201609'
    con.reqHistoricalData(symbol_id, qqq, '', '1 W', '1 day', 'TRADES', 1, 2)  

    symbol_id = symbol_id + 1  
    sleep(10)  

print (dataDownload) 
filename = 'downloaded_symbols.csv'  
csvfile = open('csv_day_test/' + filename,'w')  
for item in dataDownload:  
    csvfile.write('%s \n' % item)  
csvfile.close()

Upvotes: 2

Views: 1370

Answers (2)

newcool
newcool

Reputation: 319

Updated answer:

Documentation update to convert bar data to dataframe:

# convert to pandas dataframe
df = util.df(bars)

Upvotes: 0

Jettiesburg
Jettiesburg

Reputation: 31

Within def historical_data_handler(msg) replace the below code; this will write directly to a dataframe.

  else:
df_new = pd.DataFrame( columns=['Code', 'Date', 'Close'])
j = 0
for i in newDataList:
    df_new.loc[j] = i.split(',')
    j = j + 1
df_new.set_index('Date',inplace=True)
#print df_new

Consider writing the values msg.close directly into the df rather than first placing in a string and splitting.

Upvotes: 1

Related Questions