CODEWITHSUNDEEP
hadoophivebigdatahiveqlhadoop2
paolov
paolov

Reputation: 2307

Is it possible to create a hive table with text output format?

My first attempt was:

CREATE TABLE t1 ( 
  a string )       
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
STORED AS TEXTFILE ;

But the result of that is:

CREATE TABLE t1 ( 
  a string )                                            
ROW FORMAT DELIMITED                                            
  FIELDS TERMINATED BY ','                                      
STORED AS INPUTFORMAT                                           
  'org.apache.hadoop.mapred.TextInputFormat'                    
OUTPUTFORMAT                                                    
  'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'  ;

which seems to result in a binary file and not a text file.

I insert data into t1:

insert into t1 values ( "hello");
INFO  : Loading data to t1 
INFO  : Table t1 stats: [numFiles=1, numRows=1, totalSize=14, rawDataSize=5]
No rows affected (86.403 seconds)

The hdfs file that results is:

14 2017-10-18 17:20 t1/000000_0.deflate

And the contents are binary. What I actually need is a text file.

So, is it possible to get an hdfs output format which is text?

BTW, I am using a hortonworks big data distribution. HDP 2.5.0.

$ hdp-select | grep hive
hive-metastore - 2.5.0.0-1245
hive-server2 - 2.5.0.0-1245
hive-server2-hive2 - 2.5.0.0-1245
hive-webhcat - 2.5.0.0-1245

Upvotes: 6

Views: 8463

Answers (1)

leftjoin
leftjoin

Reputation: 38290

zlib/deflate compression format - It is the default data compression format. The file extension of this compression format is .deflate. The following configuration is used to set this format:

SET hive.exec.compress.output=true;
SET mapred.output.compression.codec=org.apache.hadoop.io.compress.DefaultCodec;

To switch-off compression use this:

SET hive.exec.compress.output=false;

And instead of specifying INPUTFORMAT, OUTPUTFORMAT you can just write simply STORED AS TEXTFILE See this answer: https://stackoverflow.com/a/44454578/2700344

Upvotes: 3

Related Questions