CODEWITHSUNDEEP
hadoophive
sjd
sjd

Reputation: 1401

Hive truncate table takes too much time

My hive query Truncate table tablename is taking too much time. Table definition has these properties defined

CLUSTERED BY(field1) INTO 2 BUCKETS 
STORED AS ORC TBLPROPERTIES('transactional'='true');

The data in table may be 20-30k rows only.

ACID transactions are enabled.

set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.enforce.bucketing=true;
set hive.exec.dynamic.partition.mode=nostrict;
set hive.compactor.initiator.on=true;
set hive.compactor.worker.threads=1;

After waiting for long. It throw an error as below

    FAILED: Error in acquiring locks: Lock acquisition for 
LockRequest(component:[LockComponent(type:EXCLUSIVE, level:TABLE, dbname:db1, 
tablename:tbl1, operationType:NO_TXN, isAcid:true)], txnid:0, user:xyz, 
hostname:host123, agentInfo:xyz_20190310220349_62d794b8-3166-4049-b9f9-646e40f1d344) timed out after 5503335ms.  LockResponse(lockid:5563, 
state:WAITING)

But no other user or job is using this table. So as to wait for the lock. What else could be the reason for the wait?

Also an insert query was executed right before the truncate(for a particular condition) .

Upvotes: 3

Views: 1424

Answers (2)

sjd
sjd

Reputation: 1401

As there was no other answer i would like to mention that Delete from table completed in usual time(took 2 mins and more importantly No lock error) in my case compared to Truncate.

Upvotes: 1

rbyndoor
rbyndoor

Reputation: 729

Hive's concurrency support for acid tables are not quite right.

As per https://community.hortonworks.com/content/supportkb/150639/hive-queries-randomly-fail-due-to-error-in-acquiri.html

Disable concurrency support with (set hive.support.concurrency=false) and restart the affected components.

Upvotes: 0

Related Questions