CODEWITHSUNDEEP
arraysjsonapache-pig
nishant
nishant

Reputation: 925

Load JSON data containing arrays in PIG

I have JSON file of the format:

{"id": "59b6808364fdb09cde10ad3b","balance": "$1,972.02","age": 35,"eyeColor": "green","tags": ["aute","nostrud","pariatur","adipisicing","irure"]}
{"id": "59b6808334cd60be95e5c166","balance": "$3,697.85","age": 32,"eyeColor": "blue","tags": ["tempor","non","ad","adipisicing","ut"]}
{"id": "59b680834544a828191abc88","balance": "$1,102.43","age": 38,"eyeColor": "brown","tags": ["quis","non","ut","veniam","ipsum"]}

I need to load this data into pig. I am using:

raw_data = LOAD '/path/to/file' USING JsonLoader('id:chararray, balance:chararray, age:int, eyeColor:chararray, tags:chararray')

I am not getting correct result using this when using dump raw_data;

What is the correct data type to load arrays in Apache PIG? There is another question which mentions how to expand an array but for my case I can have variable elements in tags element.

Even if I could covert the array to string and then load it that would be fine.

Upvotes: 1

Views: 521

Answers (1)

nobody
nobody

Reputation: 11080

Enclose the field inside tags with {}

raw_data = LOAD '/path/to/file' USING JsonLoader('id:chararray, balance:chararray, age:int, eyeColor:chararray, tags:{items:chararray}')

Upvotes: 1

Related Questions