LAP
LAP

Reputation: 61

How to add and backfill a column to a ClickHouse materialized vew and the underlying table

I'm working with a ClickHouse database and facing a challenge related to table structure modification and data backfilling. We use a materialized view to populate a table, and we need to add a new column to it. The data is inserted by an external component. Here's a simplified scenario:

CREATE MATERIALIZED VIEW IF NOT EXISTS some_mv ON CLUSTER default TO some_table
AS
SELECT A, B
FROM some_source_table
GROUP BY A, B
  1. What is the recommended procedure to modify a materialized view in scenarios like this?
  2. What are some effective strategies to evaluate and mitigate performance concerns during such migrations, particularly in cases of large table sizes? And what would be considered "large" in this context, assuming 10 to 15 columns? (es. millions of rows? 100s of millions? billions?)
  3. If I need to temporarily drop the materialized view, are there strategies to ensure that I can still have the data ingested during that time once the materialized view is recreated?
  4. What are some rollback or mitigation strategies to consider in case of unexpected issues?

Any guidance or best practices on handling this type of task would be greatly appreciated.

Thank you!

Upvotes: 1

Views: 329

Answers (0)

Related Questions