CODEWITHSUNDEEP
sqlapache-sparkhadoophiveapache-spark-sql
Arushi
Arushi

Reputation: 203

Working of Alter Table/Partition "Concatenate" Command

Wanted to know how does the Alter Table/Partition Concatenate command works, couldn't find much on https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-AlterTable/PartitionConcatenate.

Supposedly I am running this command on a particular partition, and at the same time I run a SELECT command on the same partition.

Can there be a race condition, that I attempt to read just when the concatenate command was deleting multiple files already present in that partition and had not yet replaced it with the new combined file from the temp folder it created.

Replicating this would be difficult, if anybody has any idea please help!

Upvotes: 0

Views: 150

Answers (1)

notNull
notNull

Reputation: 31540

There will be no race condition, When we are running alter table <tb_name> concatenate; there will be Exclusive lock acquired on the table (if specific partition mentioned then lock on that specific partition only).

The Exclusive Lock will be release once the MR job got finished.

Then only your Select query will be executed on the table, until the Exclusive lock on the table your select query will be just waiting in the queue.

From Official Hive Documentation:

Hive Command                                Locks Acquired
---------------------------------------     ---------------
alter table T1 partition P1 concatenate  |  EXCLUSIVE Lock on T1.P1
alter table T1 concatenate               |  EXCLUSIVE Lock on T1

To check locks on specific Hive table:

show locks <db>.<tb_name>;

Upvotes: 2

Related Questions