CODEWITHSUNDEEP
apache-pig
Keshav Pradeep Ramanath
Keshav Pradeep Ramanath

Reputation: 1687

Sum of fields across column in Pig

I have the below test data. 

A   B   C

M   O

M   M   M

M   M   M

N       O

P       N

I would also like to get the count of each of these values,like M=7, N=2, O=2, P=1. , where A,B and C are column headings. I have written the below code.

 test=  LOAD 'testdata' USING PigStorage(',') as(A:chararray,B:chararray,C:chararray); 
 values = FOREACH test GENERATE A==''?'null':(A is null?'null':A)) as A,(B==''?'null':(B is null?'null':B)) as B,(C==''?'null':(C is null?'null':C)) as C;  
 grp = GROUP values ALL;  
 A = FOREACH grp {
 B =FILTER test.A=='M' OR test.B=='M' OR test.C=='M';
 C =FILTER test.A=='N' OR test.B=='N' OR test.C=='N';
 D =FILTER test.A=='O' OR test.B=='O' OR test.C=='O';
 E =FILTER test.A=='P' OR test.B=='P' OR test.C=='P';
 GENERATE group, COUNT(B), COUNT(C),COUNT(D),COUNT(E);
  };

I am getting an error "Scalar has more than one row in the output". Any inputs would help!!

Upvotes: 0

Views: 341

Answers (1)

nobody
nobody

Reputation: 11080

Load the data as a line,tokenize the fields and then count

A = load 'testdata' as (line:chararray);
B = foreach A generate flatten(TOKENIZE((chararray)line)) as word;
C = group B by word;
D = foreach C generate group,COUNT(B);
DUMP D;

Upvotes: 1

Related Questions