CODEWITHSUNDEEP
hadoophivehadoop-partitioning
Varshini
Varshini

Reputation: 69

How does Hive partition works

Lets assume the below table:

as schema: ID,NAME,Country and my partition key is country.

If my query is like:

select * from table where id between 155555756 to 10000000000;

The partition will not work in that case, right? .

On a simple note .What if I do not use partition key in my query . So table full scan will be there, right?

Upvotes: 1

Views: 354

Answers (1)

Jahan Balasubramaniam
Jahan Balasubramaniam

Reputation: 360

Answer to your first question is yes, this query plan will not do partition pruning.

You can use the following statement to check if the query does partition pruning: explain dependency <your query>

Answer to your second question - It depends!

If the hive.mapred.mode is set to strict, then hive will not allow to do full table scans, and few other "risky" operations like cross joins etc.,

Depending on the version of hive you are using, these settings also affect the number of partitions that can be scanned by a single query

  • hive.metastore.limit.partition.request (or)
  • hive.limit.query.max.table.partition

Upvotes: 1

Related Questions