CODEWITHSUNDEEP
clickhousesharding
zzzzzz
zzzzzz

Reputation: 15

ClickHouse: copy existing data to a new cluster with a different layout

I have a ClickHouse setup running version 21.8 with 3 shards, none of them are replicated. This setup holds 92 tables occupying approximately 60G of data.

SELECT cluster, shard_num, shard_weight, replica_num FROM system.clusters

Row 1:
──────
cluster:      clickhouse
shard_num:    1
shard_weight: 1
replica_num:  1
Row 2:
──────
cluster:      clickhouse
shard_num:    2
shard_weight: 1
replica_num:  1
Row 3:
──────
cluster:      clickhouse
shard_num:    3
shard_weight: 1
replica_num:  1

I would like to change the existing layout from 3 to 2 shards.

ClickHouse doesn't support shard rebalancing, so I looked into options to achieve my goal and even though it's not performant, I was under the impression that the following approach:

  1. create a fresh setup with my desired layout
  2. copy data from the existing setup to the one via an INSERT FROM SELECT

Would be the most straightforward way to do it.

Are there better ways to do this?

Upvotes: 0

Views: 65

Answers (1)

Derek Chia
Derek Chia

Reputation: 470

INSERT SELECT to a distributed table pointing to a cluster with 2 shards would work.

Also, 21.9 is very old. Please consider upgrading.

Upvotes: 0

Related Questions