CODEWITHSUNDEEP
hadoopapache-pighadoop2
Geod Master
Geod Master

Reputation: 25

Not recognizing JARs using Apache-Pig on Hadoop

My goal is to use spatial functionality in my pig-scripts with pigeon. To use the pigeon-functions I register three JARs (pigeon-0.2.1.jar, esri-geometry-api-1.2.1.jar and jts-1.8.jar) at the beginning, which runs without errors and warnings. When I run pig commands in the pig command line (grunt) everything is fine (except some deprication warnings, but refering to other posts this can be ignorned) but as soon as I want to run a pigeon command like ST_MakePoint an error comes up:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve ST_MakePoint using imports: [, java.lang., org.apache.pig.builtin., org.apache.pig.impl.builtin.]

I am using: Ubuntu 14.04.4 LTS, Hadoop 2.7.1 (singlenode), pig 0.15.0

Any suggestions on how to fix this?

Upvotes: 0

Views: 153

Answers (1)

Taha Naqvi
Taha Naqvi

Reputation: 1766

1st way:

Call Syntax in Pig : packagename.ClassName(arg0...)

Source Code From Github :

    package edu.umn.cs.pigeon;

    import java.io.IOException;

    import org.apache.pig.EvalFunc;
    import org.apache.pig.data.DataByteArray;
    import org.apache.pig.data.Tuple;

    import com.esri.core.geometry.Point;
    import com.esri.core.geometry.SpatialReference;
    import com.esri.core.geometry.ogc.OGCPoint;

    /**
     * @author Ahmed Eldawy
     *
     */

    public class MakePoint extends EvalFunc<DataByteArray> {

      @Override
      public DataByteArray exec(Tuple input) throws IOException {
        if (input.size() != 2)
          throw new IOException("MakePoint takes two numerical arguments");
        double x = ESRIGeometryParser.parseDouble(input.get(0));
        double y = ESRIGeometryParser.parseDouble(input.get(1));
        Point point = new Point(x, y);
        OGCPoint ogc_point = new OGCPoint(point, SpatialReference.create(4326));
        return new DataByteArray(ogc_point.asBinary().array());
      }
    }

So syntax would be

FOREACH points GENERATE edu.umn.cs.pigeon.MakePoint(X,Y)

2nd Way: Use Short Function Names

To avoid writing the full function name (package + class name), you can use the 'pigeon_import.pig' file that creates a short name for all functions. You can download the file here and place it next to your Pigeon script. To load this file into your script, issue the following line to the beginning of your script. IMPORT 'pigeon_import.pig'; After that, you can write the envelope function as ST_MakePoint instead of edu.umn.cs.pigeon.MakePoint

Reference Blog

Hope this helps!!!

Upvotes: 3

Related Questions