CODEWITHSUNDEEP
hadoopmapreduceapache-pig
abhi
abhi

Reputation: 4792

whitespace character in case of parameter substitution

I want to pass a filter statement with in my pig script using parameter substitution

For that I have tried

    exec -param flt='a1==1 AND a2=2' filterscript.pig

But sadly it is throwing an exception message

    ERROR org.apache.pig.tools.grunt.Grunt - ERROR 101: Local file 'AND' does not exist.

Pig version - 0.9.2

I have tried flt='\'a1==1 AND a2=2\'' and flt="a1==1 AND a2==2" suggested by pig users in apache forum as well as seen a similar post in SO.

Any help will be appreciated

Upvotes: 0

Views: 637

Answers (2)

Lorand Bendig
Lorand Bendig

Reputation: 10650

If you run your script outside the grunt shell you can do the followings:

pig -param flt="a1\=\=1 AND a2\=\=2" -f filterscript.pig

where filterscript.pig is something like this:

A = load ...
...
B = filter A by $flt;
...

Note that the '=' is also escaped, otherwise the filter condition won't be evalued to boolean.

If you want to use the filter substitution within the grunt shell as you tried with exec, then you'll encounter the whitespace problem. Since escaping the whitespace character doesn't work, as a workaround you can create a parameter file :

cat params.txt
flt="a1\=\=1 AND a2\=\=2"

Then issue:

exec -param_file params.txt filterscript.pig

Note: I use Pig 0.12

Upvotes: 1

Rags
Rags

Reputation: 1881

I think you are using the parameter passed as it is as a condition. If so you will get an error like this. Instead you can pass them as separate paarmeters and form the condition string inside the pig script.

exec -p p1=1 -p p2=2 filterscript.pig

Inside your filterscript.pig script you can use these parameter values in condition clauses. For example

a1==$p1 AND a2=$p2

Upvotes: 1

Related Questions