CODEWITHSUNDEEP
compressionhadoophive
ablimit
ablimit

Reputation: 2361

How to get Hive working with bz2 ?

Since bz2 is splittable and saves space, I'm planning to store the table data in bzip2 format. However it seems it's not recognizing the compressed format. It was fine with gzip format. The code I used looks like this:

CREATE TABLE IF NOT EXISTS image_bzip (
    image_id STRING,
    image_feature STRING, 
    other_stuff STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION '/hivestore/bzip';

LOAD DATA LOCAL INPATH '/nfs/directory/*.bz2' OVERWRITE INTO TABLE image_bzip;

Can somebody point me what I'm missing ?

Thanks.

Upvotes: 0

Views: 3819

Answers (2)

chirag
chirag

Reputation: 11

why don't you try creating external table with bz2 compressed files on HDFS?

Upvotes: 1

ablimit
ablimit

Reputation: 2361

if we set the compression codec correctly, it works naturally.

Upvotes: 1

Related Questions