CODEWITHSUNDEEP
apache-pig
MarAja
MarAja

Reputation: 1597

I would like to transform a map into a field in a Pig Latin script

The description of my tuples of my relation (A) is as follows: {a: int, b: int, c: map[]} the map contains only one chararray but the key is not predictable. For example, a sample of my tuples is:

(1, 100, [key.152#hello])
(8, 110, [key.3000#bonjour])
(5, 103, [key.1#hallo])
(5, 103, [])
(8, 104, [key.11#buenosdias])
...

I would like to transform my relation (A) into a B relation so the B description would be: {a: int, b: int, c: chararray}

With my sample, it would give:

(1, 100, hello)
(8, 110, bonjour)
(5, 103, hallo)
(8, 104, buenosdias)
...

(I want to filter empty maps too)

Any ideas?

Thank you.

Upvotes: 0

Views: 171

Answers (1)

Bharat Jain
Bharat Jain

Reputation: 664

Though writing the UDF is the right solution, if you want to hack something quick following solution using Regex might help.

A = LOAD 'sample.txt' as (a:int, b:int, c:chararray);
B = FOREACH A GENERATE a, b, FLATTEN(STRSPLIT(c, '#', 2)) as (key:chararray, value:chararray);
C = FOREACH B GENERATE a, b, FLATTEN(STRSPLIT(value, ']', 2)) as (value:chararray, ignore:chararray);
D = FILTER C BY value is not null; 
E = FOREACH D GENERATE a, b, value;
STORE E INTO 'output/E';

For sample input 

1       100     [key.152#hello]
8       110     [key.3000#bonjour]
5       103     [key.1#hallo]
5       103     []
8       104     [key.11#buenosdias]

The above code produces following output:

1       100     hello
8       110     bonjour
5       103     hallo
8       104     buenosdias

Upvotes: 2

Related Questions