CODEWITHSUNDEEP
jsonhadoopapache-pigbigdata
BigData
BigData

Reputation: 1

customized JSON output in pig

Need customized JSON output-- (I have two files - text file and schema file)

abc.txt -

100002030,Tom,peter,eng,block 3, lane 5,california,10021

100003031,Tom,john,doc,block 2, lane 2,california,10021

100004032,Tom,jim,eng,block 1, lane 1,california,10021

100005033,Tom,trek,doc,block 2, lane 2,california,10021

100006034,Tom,peter,eng,block 6, lane 6,california,10021

abc_schema.txt (field name and position)

rollno 1

firstname 2

lastname 3

qualification 4

address1 5

address2 6

city 7

Zipcode 8

Rules-

Expected Output-

{"rollno":"100002","firstname":"Tom","lastname:"peter","qualification":"eng","Address":"block 3 lane 5 california","zipcode":"10021"}

{"rollno":"100002","firstname":"Tom","lastname:"john","qualification":"doc","Address":"block 2 lane 2 california","zipcode":"10021"}

{"rollno":"100004","firstname":"Tom","lastname:"jim","qualification":"eng","Address":"block 1 lane 1 california","zipcode":"10021"}

{"rollno":"100005","firstname":"Tom","lastname:"trek","qualification":"doc","Address":"block 2 lane 2 california","zipcode":"10021"}

{"rollno":"100006","firstname":"Tom","lastname:"peter","qualification":"eng","Address":"block 6 lane 6 california","zipcode":"10021"}

I do not wish to hardcode the fields but read from the schema file, the idea is to have reusable code. Something like looping schema file and the text file

Upvotes: 0

Views: 82

Answers (1)

genifer
genifer

Reputation: 31

A = load 'abc.txt' using PigStorage(',') as (rollno, Fname,Lname,qua,add1,add2,city,Zipcode);

B = foreach A generate rollno, Fname,Lname,qua,concate (add1,add2,city) ,Zipcode;


C= STORE B
    INTO 'first_table.json' 
    USING JsonStorage();

Hope this helps.

Upvotes: 0

Related Questions