CODEWITHSUNDEEP
apache-pig
user1084874
user1084874

Reputation: 141

Pig If Else semantics

I have something like

a = LOAD 'input-1';
b = LOAD 'input-2';
c = UNION a,b;

where input-1 is a directory and may be sometimes empty. Whenever it is empty, the UNION throws some exception as a is NULL. UNION is just one operation here, it could be any other operation like JOIN a BY $0, b BY $0, etc.

Is it possible to check the nullness of "a" in Pig, before using it any sub sequent operation ?

Upvotes: 0

Views: 133

Answers (1)

JamCon
JamCon

Reputation: 2333

You need to pre-process your inputs using the SPLIT function. There are no if/else semantics in Pig, unfortunately.

a = LOAD 'input-1';
b = LOAD 'input-2';

SPLIT a INTO a_clean IF ($0 is not null), a_dirty IF ($0 is null);
SPLIT b INTO b_clean IF ($0 is not null), b_dirty IF ($0 is null);

c = UNION a_clean, b_clean;

Upvotes: 1

Related Questions