CODEWITHSUNDEEP
hadoophivegoogle-cloud-storage
Ankit Agrahari
Ankit Agrahari

Reputation: 379

MSCK Command Throwing Error When Google Storage Set As Location In Properties

I have a external partitioned hive table whose Location is set as 'gs://xxxx'.I have added some partitions manually and for regestering that partitions to hive metastore , i ran MSCK REPAIR command which throws following error:

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask

Please let me know why this error getting generated

Upvotes: 1

Views: 2031

Answers (2)

SamShukla
SamShukla

Reputation: 53

The solution is to run Alter Table and set location to a subdirectory in gcs as given below.

ALTER TABLE TABLENAME SET LOCATION gs://crazybucketstring/schemname/tableaname/

If you are interested to understand why it is giving error with msck repair read this answer Drop Hive Table & msck repair fails with Table stored in google cloud bucket

This problem is related to gs location of your table. Even though msck repair with ignored path validation as given in other answer works but it fails to solve underlying issue.

Upvotes: 0

Saurav Bhowmick
Saurav Bhowmick

Reputation: 328

Try this - 

set hive.msck.path.validation=ignore; MSCK REPAIR TABLE table_name;

If it doesn't work check the DDL and the Partition fields. Keep in mind only Int and String is supported as partition values

Upvotes: 2

Related Questions