CODEWITHSUNDEEP
hadoop
Yao Pan
Yao Pan

Reputation: 524

how add hadoop jars to classpath?

Hadoop 2.7.3 on my mac is installed at:

/usr/local/Cellar/hadoop/2.7.3

I write a demo to read file from HDFS using java:

import java.io.*;
import java.net.URI;
import java.net.URISyntaxException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FSDataInputStream;
import org.apache.hadoop.fs.FSDataOutputStream;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;

public class HDFSTest{

public static void main(String[] args) throws IOException, URISyntaxException{

  String file= "hdfs://localhost:9000/hw1/customer.tbl";
  Configuration conf = new Configuration();
  FileSystem fs = FileSystem.get(URI.create(file), conf);
  Path path = new Path(file);
  FSDataInputStream in_stream = fs.open(path);
  BufferedReader in = new BufferedReader(new      
  InputStreamReader(in_stream));
  String s;
     while ((s=in.readLine())!=null) {
        System.out.println(s);
     }
    in.close();
    fs.close();
  }
}

When I compile the java file ,error as shown blow：

hero:Documents yaopan$ javac HDFSTest.java 
HDFSTest.java:8: error: package org.apache.hadoop.conf does not exist
import org.apache.hadoop.conf.Configuration;
                         ^
HDFSTest.java:10: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FSDataInputStream;
                       ^
HDFSTest.java:12: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FSDataOutputStream;
                       ^
HDFSTest.java:14: error: package org.apache.hadoop.fs does not exist
import org.apache.hadoop.fs.FileSystem;

I know the reason is can not find hadoop jars，how to configure that？ ^

Upvotes: 0

Views: 4367

Answers (2)

Yao Pan
Yao Pan

Reputation: 524

Just add hadoop jars to classpath:

I install hbase using homebrew on /usr/local/Cellar/hbase/1.2.2,

add all jars under /usr/local/Cellar/hbase/1.2.2/libexec/lib to classpath:

1.edit .bash_profile

sudo vim ~/.bash_profile

2.add classpath

   #set hbase lib path
export CLASSPATH=$CLASSPATH://usr/local/Cellar/hbase/1.2.2/libexec/lib/*

  1. save and exit

    wq

Upvotes: 0

Gopi Kolla
Gopi Kolla

Reputation: 974

Locate a jar file named "hadoop-common-2.7.3.jar" under your installation (i.e /usr/local/Cellar/hadoop/2.7.3) and set it in classpath or give it directly in the command line along with javac.

javac -cp "/PATH/hadoop-common-2.7.3.jar" HDFSTest.java

(replace PATH with appropriate path)

Upvotes: 1

Related Questions