CODEWITHSUNDEEP
javalinuxhadoopsolarishdfs
user1730626
user1730626

Reputation: 447

Reading a file in Java hdfs

I ran into a problem running the program on a cluster and decided to read from hdfs file in functions map and reduce. How to read line by line hdfs file and burn to read rows in ArrayList?

Upvotes: 1

Views: 3637

Answers (1)

Happy3
Happy3

Reputation: 309

Just a code snippet for demonstration：

Path path = new Path(filePath);
FileSystem fs = path.getFileSystem(context.getConfiguration()); // context of mapper or reducer
FSDataInputStream fdsis = fs.open(path);
BufferedReader br = new BufferedReader(new InputStreamReader(fdsis));
String line = "";
ArrayList<String> lines = new ArrayList<String>();
while ((line = br.readLine()) != null) {
    lines.add(line);
}
br.close();

Upvotes: 1

Related Questions