CODEWITHSUNDEEP
hadoophiveapache-pig
clairvoyant
clairvoyant

Reputation: 129

Pig: get data from hive table and add partition as column

I have a partitioned Hive table that i want to load in a Pig script and would like to add partition as column also.

How can I do that?

Table definition in Hive:

CREATE EXTERNAL TABLE IF NOT EXISTS transactions
(
column1 string,
column2 string
)
PARTITIONED BY (datestamp string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
LOCATION '/path';

Pig script:

%default INPUT_PATH '/path'

A = LOAD '$INPUT_PATH'
         USING PigStorage('|')
         AS (
         column1:chararray, 
         column2:chararray,
         datestamp:chararray  
         );

The datestamp column is not populated. Why is it so?

Upvotes: 1

Views: 2364

Answers (1)

Tariq
Tariq

Reputation: 34184

I am sorry I didn't get the part which says add partition as column also. Once created, partition keys behave like regular columns. What exactly do you need?

And you are loading the data directly from a given HDFS location, not as a Hive table. If you intend to use Pig to load/store data from/into a Hive table you should use HCatalog.

For example :

A = LOAD 'transactions' USING org.apache.hcatalog.pig.HCatLoader();

Upvotes: 1

Related Questions