CODEWITHSUNDEEP
javahadoopbigdata
user3274576
user3274576

Reputation: 27

Creating an input stream object in Hadoop for reading a file

I have a file in my Hadoop filesystem and I need to create an InputStream object in-order to pass it as an input parameter to another API as ApiMethod(inputstreamObject).

I am using below approach as mentioned in Definitive Guide to create an input stream object but does not work.

class test {

        static {    
        URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());  } 

        InputStream in = null; 
        try {  
        in = new URL("hdfs://host/path").openStream();  
        IOUtils.copyBytes(in, System.out, 4096, false);
        Object = new ApiMethod(in);
        } finally {  
        IOUtils.closeStream(in); } 

}

Please help.

Upvotes: 1

Views: 8576

Answers (1)

salmanbw
salmanbw

Reputation: 1311

If you just want to read a file from hadoop,let's try this approach. Read a file from hadoop and write it to a local path or print it on screen.

FileSystem fileSystem = FileSystem.get(conf);

Path path = new Path("/path/to/file");

FSDataInputStream in = fileSystem.open(path);
OutputStream out = new BufferedOutputStream(new FileOutputStream(
    new File(fileOnLocal)));

byte[] b = new byte[1024];
int numBytes = 0;
while ((numBytes = in.read(b)) > 0) {
    out.write(b, 0, numBytes);
}

in.close();
out.close();
fileSystem.close();

Upvotes: 5

Related Questions