CODEWITHSUNDEEP
hadoophive
fricadelle
fricadelle

Reputation: 477

Parralel insertion into hive table with non overlapping queries

Say I have to tables T1 and T2 both partitioned on (day, country) and I run two queries in parallel similar to:

INSERT OVERWRITE TABLE T1 PARTITION (day, country) SELECT * FROM T2 WHERE country='DE'
INSERT OVERWRITE TABLE T1 PARTITION (day, country) SELECT * FROM T2 WHERE country='FR'

thus, with non overlapping queries. I am not sure what is happening in this case? According to the doc https://cwiki.apache.org/confluence/display/Hive/Locking#Locking-TurnOffConcurrency it is my understanding that the second query would have to wait until the first one is done.

Upvotes: 1

Views: 2221

Answers (1)

sk7979
sk7979

Reputation: 140

I don't think that is possible because, whenever you try insert overwrite on a table in hive that table will be in exclusive lock mode... which means you can only read from that table but not add content into the table.

Please refer to the following link which describes locking mechanism in hive: https://cwiki.apache.org/confluence/display/Hive/Locking

Upvotes: 2

Related Questions