CODEWITHSUNDEEP
javahadoopbigdataapache-pig
shanks_roux
shanks_roux

Reputation: 438

Java & Pig - Is it possible to get the pig script's output into a Java variable?

I wrote a script in Pig Latin and I executed it through Java. Here is my code:

public static void main(String args[]){

    PigServer pigServer;
    try {

        Properties props = new Properties();
        props.setProperty("fs.default.name", "hdfs://localhost:8022");
        props.setProperty("mapred.job.tracker", "localhost:8021");

        System.setProperty("javax.xml.parsers.DocumentBuilderFactory", "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");

        pigServer = new PigServer(ExecType.MAPREDUCE, props);

        pigServer.registerScript("/home/training/Desktop/text_v6.pig");



    } 
    catch (ExecException e) {   e.printStackTrace(); } 
    catch (IOException e) { e.printStackTrace(); }

}

The program works perfectly and the output is this:

(2,4L)
(3,2L)
(1,1L)
...

I can potentially have millions of lines like this as an output. That's why I don't want to store the Pig's output into a file, which will be read with Java. I would like to store immediatly the results into a Java structure, like an HashMap for instance.

Is it possible ?

Thanks.

Upvotes: 1

Views: 913

Answers (1)

jkovacs
jkovacs

Reputation: 3530

You can retrieve an Iterator over all tuples in the specified alias using the openIterator method in PigServer. This is assuming you have no other dump or store statements in your pig script.

See also: http://pig.apache.org/docs/r0.10.0/api/org/apache/pig/PigServer.html#openIterator(java.lang.String)

Upvotes: 2

Related Questions