CODEWITHSUNDEEP
javahadoopmapreduce
rakeshr
rakeshr

Reputation: 1037

hadoop mapper static initialisation

I have a code fragment in which I am using a static code block to initialize a variable. 

public static class JoinMap extends 
            Mapper<IntWritable, MbrWritable, LongWritable, IntWritable> {
        .......
        public static RTree rt = null;
        static {
            String rtreeFileName = "R.rtree";
            rt = new RTree(rtreeFileName);
        }
        public void map(IntWritable key, MbrWritable mbr,Context context)
                throws IOException, InterruptedException {
                            .........               
                List elements = rt.overlaps(mbr.getRect());
                .......

        }

    }

My problem is that the variable rt in the above code fragment is not getting initialised. Can anybody suggest a fix or an alternate way to initialise the variable. I don't want to initialise it inside my map function since that slows down the entire process.

Upvotes: 1

Views: 1891

Answers (6)

Filipp Voronov
Filipp Voronov

Reputation: 4197

public static class JoinMap extends 
        Mapper<IntWritable, MbrWritable, LongWritable, IntWritable> {
    .......
    public RTree rt = null; //note that i removed static modifier

    @Override
    public void setup(Context context) throws IOException {
        //this will be executed once on each mapper before first map(..) call
        String rtreeFileName = "R.rtree";
        rt = new RTree(rtreeFileName);
    }

    public void map(IntWritable key, MbrWritable mbr,Context context)
            throws IOException, InterruptedException {
                        .........
    }

}

Upvotes: 1

abisheksampath
abisheksampath

Reputation: 416

you can override the setup() method of mapper class to initialize anything in regards with map function. This setup() method is called only once, in the beginning of every mapper,and so, it will not slow down the mapper process

Upvotes: 0

Mahender Singh
Mahender Singh

Reputation: 1421

what if we don't declare a static block and just write.

public static RTree rt =  new RTree(rtreeFileName);

because the variable is static it will be initialized once when class is loaded in memory?

Upvotes: 0

Niels Basjes
Niels Basjes

Reputation: 10652

I'm no Java expert but it look like you have two "static" assignments for the variable rt: You have:

public static RTree rt = null;

AND

rt = new RTree(rtreeFileName);

In which order are these assigments done?

Try this instead and see if it helps

    public static final RTree rt = new RTree("R.rtree");

As far as my Java knowledge goes this assures you'll have only single assignment.

Upvotes: 0

Ryan Fernandes
Ryan Fernandes

Reputation: 8536

Is there some reason you don't want to initialize rt like this?:

public static RTree rt = new RTree("R.rtree");

Upvotes: 0

Sean Owen
Sean Owen

Reputation: 66896

That really isn't possible, unless Java itself is broken. The static initializers always fire at class load time.

Maybe whatever you're observing has an alternate explanation, like, something is setting rt back to null. Or, are you observing rt in other statically-initialized expression above? it would see null until rt's initialization finishes.

But, nevertheless, I'd say it's tidier to override the setup() method (configure() in the old API) and do initialization there. It'll happen once.

Upvotes: 2

Related Questions