CODEWITHSUNDEEP
apache-pig
systemdebt
systemdebt

Reputation: 4941

Filter columns in Pig script

I am loading data in Pig from a CSV. After having loaded data, I need to filter out columns .

exportAllProductsCleaned = FOREACH exportAllProducts 
generate $0, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $26, $27, $28, $29, $30, $31, $32, $33

Is there a way wherein I can specify only

  1. The columns I need to remove OR
  2. The range of columns I need for ex. $1-15 and then $18 - $30

Is it possible?

Upvotes: 0

Views: 909

Answers (1)

nobody
nobody

Reputation: 11080

Yes, you can do so using '..' convention.Refer Support project range expression

exportAllProductsCleaned = FOREACH exportAllProducts GENERATE $0, $1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $26, $27, $28, $29, $30, $31, $32, $33;
exportAllProductsFiltered = FOREACH exportAllProductsCleaned GENERATE $1 .. $15,$18 .. $30;

Upvotes: 1

Related Questions