CODEWITHSUNDEEP
hive
Vinay
Vinay

Reputation: 1651

How to get the partition column name of a Hive table

I'm developing a unix script where I'll be dealing with Hive tables partitioned by either column A or column B. I'd like to find on what column a table is partition on so that I can do subsequent operations on those partition instances.

Is there any property in Hive which returns the partition column directly?

I'm thinking I'll have to do a show create table and extract the partition name somehow if there isn't any other way possible.

Upvotes: 7

Views: 17380

Answers (4)

geosmart
geosmart

Reputation: 666

 List<String> parts = new ArrayList<>();
        try {
            List<FieldSchema> partitionKeys = client.getTable(dbName, tableName).getPartitionKeys();
            for (FieldSchema partition : partitionKeys) {
                parts.add(partition.getName());
            }
        } catch (Exception e) {
            throw new RuntimeException("Fail to get Hive partitions", e);
        }

Upvotes: 0

leon
leon

Reputation: 11

#use python pyhive:
import hive_client

def get_partition_column(table_name):
    #hc=hive connection
    hc=hive_client.HiveClient()
    cur=hc.query("desc "+table_name)
    return cur[len(cur)-1][0]


#################
hive_client.py

from pyhive import hive
default_encoding = 'utf-8'
host_name = 'localhost'
port = 10000
database="xxx"

class HiveClient:
    def __init__(self):
        self.conn = hive.Connection(host=host_name,port=port,username='hive',database=database)
    def query(self, sql):
        cursor = self.conn.cursor()
        #with self.conn.cursor() as cursor:
        cursor.execute(sql)
        return cursor.fetchall()

    def execute(self,sql):
        #with self.conn.cursor() as cursor:
        cursor = self.conn.cursor()
        cursor.execute(sql)

    def close(self):`enter code here`
        self.conn.close()

Upvotes: 0

Karthik
Karthik

Reputation: 41

Through scala/java api, we can get to the hive meta store and get the partition column names org.apache.hadoop.hive.metastore.HiveMetaStoreClient

val conf = new Configuration()
conf.set("hive.metastore.uris","thrift://hdppmgt02.domain.com:9083")
val hiveConf = new HiveConf(conf, classOf[HiveConf])
val metastoreClient = new HiveMetaStoreClient(hiveConf)

metastoreClient.getTable(db, tbl).getPartitionKeys.foreach(x=>println("Keys : "+x))

Upvotes: 4

Aditya
Aditya

Reputation: 2415

May be not the best, but one more approach is by using describe command

Create table:

create table employee ( id int, name string ) PARTITIONED BY (city string);

Command: 

hive -e 'describe formatted employee'  | awk '/Partition/ {p=1}; p; /Detailed/ {p=0}'

Output:

# Partition Information
# col_name              data_type               comment

city                    string

you can improve it as per your need.

One more option which i dint explore is by querying meta-store repository tables to get the partition column information for a table.

Upvotes: 4

Related Questions