user6546116
Reputation:
Maximum number of partitions in hive
I have 1500 partition in my hive tables but while doing query it is taking more time then expected. Maximum number of partitions can be created in hive table.
Upvotes: 2
Views: 7655
Answers (1)
Mayank Porwal
Reputation: 34046
Current Hive versions with RDBMS metastore backend should be able to handle 10000+ partitions.
For numerous reasons, the community is moving away from this design to leverage HBase for the metastore. Follow https://issues.apache.org/jira/browse/HIVE-9452 .
Overall design document is available here: https://issues.apache.org/jira/secure/attachment/12697601/HBaseMetastoreApproach.pdf
Upvotes: 3
Related Questions
- Max number of partitions that can be created on bigquery table
- partitions in hive interview questions
- How does Hive partition works
- Record count for Hive partitioned table
- Can we make a table having both partitioning and bucketing in hive?
- hive external partitioned table
- Many Partitions in Hive
- how to remove the limit of partitions per node in Hive?
- Hive - Bucketing and Partitioning
- Partitioning in Hive table