CODEWITHSUNDEEP
timescaledb
nehz
nehz

Reputation: 2182

Timescaledb compression, segmentby and chunking

Does the segementby col setting in timescaledb ensure that it will create a new chunk for each different value of col ? This is implied in the documentation but isn't explicitly stated anywhere.

For example, using the example from documentation will the following segmentby deviceid

time    device_id   cpu disk_io energy_consumption
[12:00:02, 12:00:01]    1   [88.2, 88.6]    [20, 25]    [0.8, 0.85]
[12:00:02, 12:00:01]    2   [300.5, 299.1]  [30, 40]    [0.9, 0.95]

Does this create 2 separate chunks for the given time range? This is important as compressed chunks cannot be written/updated to, so if for the same time range a new device id data point appears later for ingesting, would this be an issue?

Upvotes: 1

Views: 1206

Answers (1)

k_rus
k_rus

Reputation: 3219

A compressed hypertable is chunked in the same way as the original hypertable. The compression is applied to each chunk and it creates a compressed chunk for each original chunk.

segmentby specifies how to combine or group rows for compression. Each unique value of segmentby columns results in a record in the compressed chunk where values of all other columns are compressed together.

In the question example, there are two unique values in segmentby column device_id: 1 and 2. Then the corresponding values in other columns are grouped together and compressed. So in the example, there are two rows for device_id = 1 and two rows for device_id = 2. If all four original records were stored in the same chunk in the original hypertable, the two corresponding compressed records will be stored in one compressed chunk.

This is important as compressed chunks cannot be written/updated to, so if for the same time range a new device id data point appears later for ingesting, would this be an issue?

It is not supported to do any changes to the original chunk, which was compressed. So inserting data with new segmentby value will fail with an error.

UPDATE: To update data in compressed chunks, e.g., backfill data, it is necessary to manually decompress chunks, update data and then compress data back as described in decompression chunk documentation.

Upvotes: 4

Related Questions