CODEWITHSUNDEEP
hadoopapache-pig
Chaos
Chaos

Reputation: 11721

How to load a file in Pig that has a variable number of fields

I have an input file which contains records as follows:

Movie1     Actor 1, Actor 2, Actor 3, ......, Actor n
Movie2     Actor 1, Actor 2,.......Actor n

I want to load this data into a bag in Pig

 movies = LOAD 'movies.imdb' AS (......);

I am not sure how to fill up my "AS" field because the records in my input file may have variable number of fields.

Upvotes: 0

Views: 1336

Answers (1)

seedhead
seedhead

Reputation: 3805

You could just do this:

movies = LOAD 'movies.imdb' USING PigStorage(',');

If the records don't have fixed fields, using AS won't be possible

Upvotes: 2

Related Questions