CODEWITHSUNDEEP
javaparsinghadoophbasenutch
Katka
Katka

Reputation: 324

Error when creating HTable

I'm using Nutch 1.8 to crawl data from website. I am writing now a custom plugin for Nutch to parse HTML and save data to HBase. By tutorials I create configuration:

Configuration conf = HBaseConfiguration.create();

Then I call openz() method to set configuration things etc.

public static void openz() throws IOException {
    LOG.info("openz()");
    System.out.println("openz()");
    System.out.println("Establishing connection with database..");
    conf = HBaseConfiguration.create();
    conf.set("hbase.master", SERVER_IP);
    conf.set("hbase.zookeeper.quorum", MASTER_PC);
    conf.set("zookeeper.znode.parent", ZOOKEEPER_PARENT_NODE);
    conf.set("hbase.zookeeper.property.clientPort","2181");
    System.out.println("Conf here? :" + conf);
    System.out.println("Creating table variable..");
    table = new HTable(conf, "bstore");
}

At this point, on line, where I create HTable, I get

java.io.IOException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.hbase.client.ConnectionManager.createConnection java.lang.NoSuchMethodError: org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Result$Builder.setStale(Z)Lorg/apache/hadoop/hbase/protobuf/generated/ClientProtos$Result$Builder;

Previously I got a lot of trouble with libraries. When I run my plugins code on netbeans with remote connection, it works fine. Saves website data to hbase without having any trouble. But these errors I get when I launch Nutch crawler on cluster.

Upvotes: 0

Views: 1130

Answers (1)

Katka
Katka

Reputation: 324

I will post my solution to this problem maybe it will help someone in the future. I use Hadoop 2.4.0, HBase 0.98.0, Apache Nutch 1.8 and Solr 4.2.1. Problems were caused by lack of some libraries. I'll post a list with all libraries, which I inserted in nutch /lib folder (this way isn't best one because nutch isn't correctly installed on the cluster, yet).

  • activation-1.1.jar
  • apache-nutch-1.8.jar
  • asm-3.1.jar
  • avro-1.7.4.jar
  • commons-beanutils-1.7.0.jar
  • commons-beanutils-core-1.8.0.jar
  • commons-cli-1.2.jar
  • commons-codec-1.4.jar
  • commons-collections-3.2.1.jar
  • commons-configuration-1.6.jar
  • commons-digester-1.8.jar
  • commons-el-1.0.jar
  • commons-httpclient-3.1.jar
  • commons-io-2.4.jar
  • commons-lang-2.6.jar
  • commons-logging-1.1.1.jar
  • commons-math-2.1.jar
  • commons-net-1.4.1.jar
  • crawler-commons-0.3.jar
  • elasticsearch-0.90.1.jar
  • guava-11.0.2.jar
  • hadoop-auth-2.2.0.jar
  • hadoop-common-2.2.0.jar
  • hadoop-yarn-api-2.2.0.jar
  • hadoop-yarn-common-2.2.0.jar
  • hadoop-mapreduce-client-common-2.2.0.jar
  • hadoop-mapreduce-client-core-2.2.0.jar
  • hadoop-mapreduce-client-jobclient-2.2.0.jar
  • hadoop-mapreduce-client-shuffle-2.2.0.jar
  • hbase-client-0.98.0-hadoop2.jar
  • hbase-common-0.98.0-hadoop2.jar
  • hbase-protocol-0.98.0-hadoop2.jar
  • htrace-core-2.04.jar
  • httpclient-4.1.1.jar
  • httpcore-4.1.jar
  • icu4j-4.0.1.jar
  • jackson-core-asl-1.8.8.jar
  • jackson-jaxrs-1.7.1.jar
  • jackson-mapper-asl-1.8.8.jar
  • jackson-xc-1.7.1.jar
  • jasper-compiler-5.5.12.jar
  • jasper-runtime-5.5.12.jar
  • jaxb-api-2.2.2.jar
  • jaxb-impl-2.2.3-1.jar
  • jersey-core-1.8.jar
  • jersey-json-1.8.jar
  • jersey-server-1.8.jar
  • jettison-1.1.jar
  • jetty-6.1.26.jar
  • jetty-client-6.1.22.jar
  • jetty-sslengine-6.1.22.jar
  • jetty-util-6.1.26.jar
  • jsp-2.1-6.1.14.jar
  • jsp-api-2.1-6.1.14.jar
  • jsr305-1.3.9.jar
  • junit-3.8.1.jar
  • log4j-1.2.15.jar
  • lucene-analyzers-common-4.3.0.jar
  • lucene-codecs-4.3.0.jar
  • lucene-core-4.3.0.jar
  • lucene-grouping-4.3.0.jar
  • lucene-highlighter-4.3.0.jar
  • lucene-join-4.3.0.jar
  • lucene-memory-4.3.0.jar
  • lucene-queries-4.3.0.jar
  • lucene-queryparser-4.3.0.jar
  • lucene-sandbox-4.3.0.jar
  • lucene-spatial-4.3.0.jar
  • lucene-suggest-4.3.0.jar
  • netty-3.6.6.Final.jar
  • oro-2.0.8.jar
  • protobuf-java-2.5.0.jar
  • servlet-api-2.5-6.1.14.jar
  • slf4j-api-1.6.6.jar
  • slf4j-log4j12-1.6.1.jar
  • spatial4j-0.3.jar
  • stax-api-1.0-2.jar
  • tika-core-1.5.jar
  • xercesImpl-2.9.1.jar
  • xml-apis-1.3.04.jar
  • xmlenc-0.52.jar
  • xmlParserAPIs-2.6.2.jar
  • zookeeper-3.4.6.jar

Upvotes: 1

Related Questions