CODEWITHSUNDEEP
hivehive-partitions
Dipak
Dipak

Reputation: 39

Delete partition directories from HDFS, would it reflect in hive table?

Lets say I created a hive table with partition column as year, month and day and if i delete the partition from hdfs, then result get reflected in hive table or not

Upvotes: 1

Views: 5116

Answers (2)

Ani Menon
Ani Menon

Reputation: 28199

Hive table will still show the partitions, you will have to either drop the partitions deleted on HDFS manually (or drop and re-create table) and run MSCK.

Commands:

If you intend to alter the table and drop all deleted partitions-

ALTER TABLE table_name DROP [IF EXISTS] PARTITION partition_spec[, PARTITION partition_spec, ...]
  [IGNORE PROTECTION] [PURGE];            -- (Note: PURGE available in Hive 1.2.0 and later, IGNORE PROTECTION not available 2.0.0 and later)

I would go with drop and re-create table then run MSCK.

To add all existing partitions to table-

msck repair table <table_name>

Alternatively, you could drop all partitions using ALTER TABLE and then run the MSCK command.

Upvotes: 0

David דודו Markovitz
David דודו Markovitz

Reputation: 44921

Yes. The partition data will be gone.
The metastore will still hold the partition information (metadata) and you can see it using show partition mytable.
You can find the partitions need to be dropped using msck repair mytable.
You can drop the partitions using alter table mytable drop partition (...)

Upvotes: 1

Related Questions