CODEWITHSUNDEEP
hadoopgroup-byapache-pigbag
user398384
user398384

Reputation: 1204

get things out of bag in pig

In the pig example:

A = LOAD 'student.txt' AS (name:chararray, term:chararray, gpa:float);

DUMP A;
(John,fl,3.9F)
(John,wt,3.7F)
(John,sp,4.0F)
(John,sm,3.8F)
(Mary,fl,3.8F)
(Mary,wt,3.9F)
(Mary,sp,4.0F)
(Mary,sm,4.0F)

B = GROUP A BY name;

DUMP B;
(John,{(John,fl,3.9F),(John,wt,3.7F),(John,sp,4.0F),(John,sm,3.8F)})
(Mary,{(Mary,fl,3.8F),(Mary,wt,3.9F),(Mary,sp,4.0F),(Mary,sm,4.0F)})

C = FOREACH B GENERATE A.name, AVG(A.gpa);

DUMP C;
({(John),(John),(John),(John)},3.850000023841858)
({(Mary),(Mary),(Mary),(Mary)},3.925000011920929)

The last output A.name is a bag. How can I get things out of bag:

(John, 3.850000023841858)
(Mary, 3.925000011920929)

Upvotes: 3

Views: 1911

Answers (1)

Donald Miner
Donald Miner

Reputation: 39893

GROUP creats a magical item called group, which is what you grouped on. This is made for exactly this purpose.

B = GROUP A BY name;

C = FOREACH B GENERATE group AS name, AVG(A.gpa);

Check out DESCRIBE B;, you'll see that group is in there. It is a single value that represents what was in the BY ... part of the GROUP.

Upvotes: 4

Related Questions