CODEWITHSUNDEEP
opentsdb
user1676605
user1676605

Reputation: 1447

Importing csv data into OpenTSDB

I have successfully installed OpenTSDB ontop of a Cloudera Hadoop/HBase cluster.

My question is, I have reams of historical 1 minute stock data that looks like this:

"Date","Time","Open","High","Low","Close","Volume" 
12/30/2002,0930,24.53,24.65,24.53,24.65,762200
12/30/2002,0931,24.65,24.68,24.52,24.6,90400

.....

From the documentation in the QS Guide, it says in the Batch imports section:

./tsdb import your-file

When I try this on my data it throws an unhelpful exception.

Any hints on how to import this into OpenTSDB? Thanks.

Upvotes: 0

Views: 2224

Answers (2)

soelu
soelu

Reputation: 574

I wrote a small csv importer for opentsdb.

https://github.com/soeren-lubitz/csv-to-opentsdb

It works for CSV-Files in form of

Timestamp,Foo,Bar
1483342774,42.1,23.2

Hope this helps. Feedback would be appreciated.

Upvotes: 0

tsuna
tsuna

Reputation: 1846

You need to write a script to transform your CSV in something that is in the OpenTSDB format. The general format for OpenTSDB is metric timestamp value tags

For instance your sample could be written as follows:

stock.open 1041269400 24.53 symbol=XXX
stock.high 1041269400 24.65 symbol=XXX
stock.low 1041269400 24.53 symbol=XXX
stock.close 1041269400 24.65 symbol=XXX
stock.volume 1041269400 762200 symbol=XXX
stock.open 1041269460 24.65 symbol=XXX
stock.high 1041269460 24.68 symbol=XXX
stock.low 1041269460 24.52 symbol=XXX
stock.close 1041269460 24.6 symbol=XXX
stock.volume 1041269460 90400 symbol=XXX

Although since it appears that you're working with 1-minute periods, the open/close is redundant, so maybe this would be more appropriate:

stock.quote.1m 1041269340 24.53 symbol=XXX
stock.quote.1m 1041269400 24.65 symbol=XXX
stock.quote.1m 1041269460 24.6 symbol=XXX

Upvotes: 4

Related Questions