Symptom:
Reading table blocks creating tables in the same database when hive.support.concurrency=true.Creating table will fail with below errors due to database level lock.
hive> create table test(id int); conflicting lock present for default mode EXCLUSIVE
Env:
Hive 0.13Root Cause:
By default in Hive 0.13 when hive.support.concurrency=true, it is usinghive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DummyTxnManager hive.lock.manager=org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManagerSome DDLs needs Exclusive lock on database using this transaction manager.
This issue is reported in HIVE-9199 which is fixed in Hive 1.1.
Minimum reproduce:
1. Set hive.support.concurrency=true in hive-site.xml.2. Open 2 hive sessions:
--Session A: create table passwords (col0 string, col1 string, col2 string, col3 string, col4 string, col5 string, col6 string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ":"; load data local inpath "/etc/passwd" into table passwords; select count(*) from passwords a,passwords b, passwords c, passwords d where a.col0=b.col0 and a.col0=c.col0 and a.col0=d.col0; --Session B: hive> create table test(id int); conflicting lock present for default mode EXCLUSIVE
Solution:
This is current lock behavior in Hive 0.13. You can disable this feature by setting hive.support.concurrency=false in hive-site.xml.Or upgrade to Hive 0.14 or above in which new lock manager DbLockManager and new transaction manager DbTxnManager are introduced.
F9C79A47
ReplyDeleteKarabük
Osmaniye
Kırşehir
Artvin
Bayburt
Bolu
Tunceli
Batman
Ağrı
9B0E454A
ReplyDeleteAntep
Bolu
Çanakkale
Mardin
Maraş
Uşak
Kastamonu
Sinop
Giresun