Reputation: 9
I am trying to read a file from HDFS using following code:
final Configuration configuration = new Configuration();
configuration.set("fs.defaultFS", "hdfs://localhost:8020/user/training/");
FileSystem fileSystem = FileSystem.get(configuration);
String filePath = "hdfs" + "://" + "localhost:8020" + "/user/training/test.txt";
File fileToProcess = new File(filePath); // path of file
FileInputStream fis = new FileInputStream(fileToProcess);
The last statement of the program is returning following exception:
Exception in thread "main" java.io.FileNotFoundException:
hdfs:/localhost:8020/user/training/test.txt (No such file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
Any idea what could be issue? File is present at the location. Only difference that I could note is : Although I have put "//" in the URL, the last statement is omitting one slash while creating the input stream.
This code is being called from MR Driver class.
Upvotes: 0
Views: 1842
Reputation: 5891
Configuration configuration = new Configuration();
FileSystem fs = FileSystem.get(new URI("hdfs://localhost:8020"),
configuration);
Path path = new Path("hdfs://localhost:8020/user/training/test.txt");
System.out.println(fs.exists(path));
FSDataInputStream fin = fs.open(path);
Upvotes: 1
Reputation: 449
Tapan, Configuration Parameter fs.defaultFS must denotes only Namenode hostname .Why it has the value of the HDFS directory itself.Can you try removing it ?
Upvotes: 0