HBase map/reduce dependency issue

Question

1. Overview

---

I developed a Rest api service based on resteasy framework. In the service, i will store data to HBase database. then, execute map/reduce process trigged by some condition(e.g. insert one record).

2. Requires

---

In the Map class, i import some third part libraries. i do not want to package those libraries to the war file.

TableMapReduceUtil.initTableMapperJob(HBaseInitializer.TABLE_DATA,   // input HBase table name
                                          scan,                      // Scan instance to control CF and attribute selection
                                          LuceneMapper.class,        // mapper
                                          null,                      // mapper output key
                                          null,                      // mapper output value 
                                          job);
FileOutputFormat.setOutputPath(job, new Path("hdfs://master:9000/qin/luceneFile"));
job.submit();

3. Problem

---

If package all libraries in the war file which will be deploy to jetty container, it work well. if not package third part library to the war,but upload those library to hdfs and add them to class path, it does not work. like below

conf.set("fs.defaultFS","hdfs://master:9000"); 
FileSystem hdfs = FileSystem.get(conf); 
Path classpathFilesDir = new Path("bjlibs"); 
FileStatus[] jarFiles = hdfs.listStatus(classpathFilesDir); 
for (FileStatus fs : jarFiles) { 
      Path disqualified = new Path(fs.getPath().toUri().getPath()); 
      DistributedCache.addFileToClassPath(disqualified, conf);
}
hdfs.close();

Answer 1

try TableMapReduceUtil.addHBaseDependencyJars()