CODEWITHSUNDEEP
javahivehive-metastore
xingbin
xingbin

Reputation: 28289

How to check if a partition exists in Hive?

I have a Hive table, which is partitioned by column dt. I need to add a partition if it does not exists, for exmaple, dt='20181219'.

Now I'm using HiveMetaStoreClient#getPartition(dbName, tableName, 20181219). If the partition does not exists, then catch NoSuchObjectException and add it.

Is there any elegant way to achieve this in Java?

Upvotes: 8

Views: 1651

Answers (1)

Stephen C
Stephen C

Reputation: 719576

Use add_partition(Partition, ifNotExists, needResults) (javadoc) ... which (if the 2nd argument is true) will only create an partition if it doesn't already exist.

Alternatively, just use add_partition(Partition) to add the partition without a test, and catch the AlreadyExistsException if it occurs.

Any scheme that involves testing that an action is possible and then doing that action has a potential race condition. In between the "test" and the "do", some other agent (thread, client, whatever) could do an action (the same one or a different one) that will cause your attempt to fail.

So your current approach is not only ugly. It also has a potential race condition if partitions could be created by multiple agents.

Upvotes: 6

Related Questions