CODEWITHSUNDEEP
hivehiveql
Karthik
Karthik

Reputation: 1811

hive tables with parquet data format reorder columns

We are using hive 1.1.0, have a requirement to reorder columns for one of huge table with 100's of columns for user readability. But when we reorder columns in table its failing with below error.

Alternative to reorder columns is create a view on table. 

hive> desc test_parquet;
OK
name                    string
age                     int
dept                    string
salary                  string
city                    string

# Partition Information
# col_name              data_type               comment

city                    string
Time taken: 0.053 seconds, Fetched: 10 row(s)
hive> ALTER TABLE test_parquet REPLACE COLUMNS (age int,name string, dept string, salary string);
OK
Time taken: 0.451 seconds
hive> desc test_parquet;
OK
age                     int
name                    string
dept                    string
salary                  string
city                    string

# Partition Information
# col_name              data_type               comment

city                    string
Time taken: 0.051 seconds, Fetched: 10 row(s)
hive> select * from test_parquet;
OK
Failed with exception java.io.IOException:java.lang.UnsupportedOperationException: Cannot inspect org.apache.hadoop.io.IntWritable
Time taken: 0.121 seconds

Upvotes: 1

Views: 2408

Answers (2)

jagath
jagath

Reputation: 238

This post is a bit old, however, I thought it would benefit others if have a similar issue!

You can try the below change column method that wors:

ALTER TABLE test_parquet CHANGE COLUMN age age int FIRST; which moves the column to first otherwise, if you want to re order after certain column, below would help:

ALTER TABLE test_parquet CHANGE COLUMN name name string AFTER age;

Upvotes: 0

David דודו Markovitz
David דודו Markovitz

Reputation: 44921

ALTER TABLE ... REPLACE COLUMNS works on the metadata level (metastore).
The new columns order does not match the actual data.
For parquet you get an exception, for textfile you would get NULL values.
You should go with your alternative solution - using a view.

Upvotes: 1

Related Questions