CODEWITHSUNDEEP
apache-sparkhive
Regressor
Regressor

Reputation: 1973

How to perform MSCK REPAIR TABLE to load only specific partitions

I have data in AWS S3 for more than 2 months that is partitioned and stored by day. I want to start using the data using the external table that I created.

Currently I see only a couple of partitions and I want to make sure my metadata picks up all the partitions. I tried using msck repair table tablename using hive after logging in to EMR Cluster's master node. However, may be due to data volume, it is taking a lot of time to execute that command.

Can I do msck repair table so that I can load only a specific day? does msck allow to load specific partitions?

Upvotes: 1

Views: 1896

Answers (1)

mazaneicha
mazaneicha

Reputation: 9427

You can use

ALTER TABLE table_name ADD [IF NOT EXISTS] PARTITION partition_spec [LOCATION 'location'][, PARTITION partition_spec [LOCATION 'location'], ...];

...as described in Hive DDL doc.

Upvotes: 1

Related Questions