CODEWITHSUNDEEP
csvhadoopapache-pigcloudera
Ojas Kale
Ojas Kale

Reputation: 2159

Is there any way toread only required columns from CSV in Pig?

I have a CSV file with about 50 columns in it. I want to read only particular columns from it. Is there any way I can do that?

Example:

CSV has, col1, col2, col3, col4, col5, col6,......,col50

I want load 

load 'path to file' as (col1, col18, col47);

I want to do something like that.

Upvotes: 1

Views: 479

Answers (1)

kecso
kecso

Reputation: 2485

Load it than project it:

REGISTER 'piggybank.jar'
DEFINE CSVLoader org.apache.pig.piggybank.storage.CSVLoader();
A = LOAD 'data.csv' USING CSVLoader();
B = FOREACH A GENERATE 
    $0 AS col1,
    $17 AS col18,
    $46 AS col47;
...

Upvotes: 1

Related Questions