CODEWITHSUNDEEP
timescaledb
MSIS
MSIS

Reputation: 322

Select the right time chunk with TimescaleDB

In a system where data is being pushed almost every ms what is the best size of time buckets to use?

Queries are usually made for only a subset of the data (depending on the key), and span from minutes to hours.

Upvotes: 0

Views: 2066

Answers (1)

k_rus
k_rus

Reputation: 3219

It is recommended to set the size of the chunk, so it is 25% of the memory including the data and indexes. So all inserts will go to hot data in memory.

From the suggestions on Timesacle docs:

The key property of choosing the time interval is that the chunk (including indexes) belonging to the most recent interval (or chunks if using space partitions) fit into memory. As such, we typically recommend setting the interval so that these chunk(s) comprise no more than 25% of main memory.

TIP:Make sure that you are planning for single chunks from all active hypertables fit into 25% of main memory, rather than 25% per hypertable.

To estimate the size of a chunk it is important to take in account number of columns and average size of the values, also size of values in indexes in addition to the rate. Also from the doc:

To determine this, you need to have a general idea of your data rate. If you are writing roughly 2GB of data per day and have 64GB of memory, setting the time interval to a week would be good. If you are writing 10GB per day on the same machine, setting the time interval to a day would be appropriate.

The size of chunks can be changed and will affect only new chunks. Too large chunk might affect ingesting rate until the time period is fully completed and new chunk sizes are used.

Upvotes: 2

Related Questions