CODEWITHSUNDEEP
pythoncryptocurrencyhistorical-dbkucoin
John
John

Reputation: 29

KuCoin API error for historical data in python

I need to download 1min kline data from 2019 from Kucoin API. It allows to get 1500 data points per request, so I created a function to get the length I need in chunks. The problem is that for some chunks I get an error.

KucoinAPIException(<Response [429]>)

which is "too many requests" error. From what I read online limit is 30 calls in 10 seconds, I am doing 1 in 10 seconds, so not too sure why is there this error? When I did time.sleep(0) I got a temp block as expected.

How to get lots of data whilst being kind to API? Where is the error in the code?

def get_ku_his(sym, tf, str_start, str_end):
    start_ts = int(datetime.strptime(str_start, '%Y-%m-%d %H:%M').replace(tzinfo=timezone.utc).timestamp())
    if str_end==0: 
        end_ts=0
    else:
        end_ts = int(datetime.strptime(str_end, '%Y-%m-%d %H:%M').replace(tzinfo=timezone.utc).timestamp())
    data = kuclient.get_kline_data(sym, tf, start_ts, end_ts)
    data = pd.DataFrame(data, columns = ['timestamp', 'open', 'close', 'high', 'low', 'transaction amount', 'volume'])
    data['timestamp'] = data['timestamp'].astype(float)*1000
    data['timestamp'] = pd.to_datetime(data['timestamp'], unit='ms')
    data.set_index('timestamp', inplace=True)
    data = data[["close","volume"]]
    data.dropna(inplace=True)
    data = data.astype(float)
    data = data.sort_index()
    return data

def get_ku_hp(sym, tf, str_start):
    data = get_ku_his(sym, tf, str_start, 0)
    temp_start = data.index[0].strftime('%Y-%m-%d %H:%M')
    while str_start < temp_start:
        try:
            temp = get_ku_his(sym, tf, str_start, temp_start)
            print("Downloaded part from %s to %s" %(str_start, temp_start))
            temp_start = temp.index[0].strftime('%Y-%m-%d %H:%M')
            data = data.append(temp)
            time.sleep(10)
        except Exception as e: print(repr(e))
    data = data.sort_index()
    print("%s Downloaded %s data for %s from %s to %s" % (datetime.today().strftime("%H:%M:%S"), tf, sym, data.index[0], data.index[-1]))
    return data

test = get_ku_hp("AIOZ-USDT","1min","2019-01-05 22:00")

Console messsages:

*Downloaded part from 2019-01-05 22:00 to 2021-10-19 11:45
Downloaded part from 2019-01-05 22:00 to 2021-10-18 10:45
Downloaded part from 2019-01-05 22:00 to 2021-10-17 09:45
KucoinAPIException(<Response [429]>)
Downloaded part from 2019-01-05 22:00 to 2021-10-16 08:45
Downloaded part from 2019-01-05 22:00 to 2021-10-15 07:45
Downloaded part from 2019-01-05 22:00 to 2021-10-14 06:45
Downloaded part from 2019-01-05 22:00 to 2021-10-13 05:45
Downloaded part from 2019-01-05 22:00 to 2021-10-12 04:45
Downloaded part from 2019-01-05 22:00 to 2021-10-11 03:45
Downloaded part from 2019-01-05 22:00 to 2021-10-10 02:45
Downloaded part from 2019-01-05 22:00 to 2021-10-09 01:45
Downloaded part from 2019-01-05 22:00 to 2021-10-08 00:45
Downloaded part from 2019-01-05 22:00 to 2021-10-06 23:45
Downloaded part from 2019-01-05 22:00 to 2021-10-05 22:45
Downloaded part from 2019-01-05 22:00 to 2021-10-04 21:45
Downloaded part from 2019-01-05 22:00 to 2021-10-03 20:45
KucoinAPIException(<Response [429]>)
KucoinAPIException(<Response [429]>)
Downloaded part from 2019-01-05 22:00 to 2021-10-02 19:45
Downloaded part from 2019-01-05 22:00 to 2021-10-01 18:45
Downloaded part from 2019-01-05 22:00 to 2021-09-30 17:45*

Upvotes: 1

Views: 1501

Answers (1)

TheQwertyipad
TheQwertyipad

Reputation: 28

To do a lot of requests try to use Websocket, here is a lib with a documentation. Or you can change requests IP using a VPN or Proxy.

Upvotes: 0

Related Questions