CODEWITHSUNDEEP
hadoopmapreducehdfs
Ahmadov
Ahmadov

Reputation: 1607

Copying files from HDFS to local file system with JAVA

I am trying to copy files from HDFS to local filesystem for preprocessing. The below code should work according to the documentation. Although it doesn't give any error messages and the mapreduce job runs smoothly I can not see any output on my local hard drive. What do you think the problem is? Thanks. 

        try {
            Path phdfs_input = new Path("hdfs://master:54310/user/hduser/conninput/"+value.toString());
            Path plocal_input = new Path("/home/hduser/Desktop/"+avlue.toString());
            FileSystem fs = FileSystem.get(context.getConfiguration());
            fs.copyToLocalFile(phdfs_input, plocal_input);

           /* String localoutput_file = "/home/hduser/Destop/output/"+value.toString();
            String cmd1[] = {"mafia", "-mfi", ".5", "-ascii", "~/Desktop/"+value.toString(), localoutput_file };
            File mafia_dir = new File("/home/hduser/");
            ShellCommandExecutor s = new ShellCommandExecutor(cmd1, mafia_dir);*/
        } catch (Exception e) {
            e.printStackTrace();
        }

Upvotes: 2

Views: 5619

Answers (1)

Hari Menon
Hari Menon

Reputation: 35495

Try using /user/hduser/conninput/"+value.toString() in the Path constructor instead of providing the master:54310 part. It should figure out master:54310 from the Configuration.

Upvotes: 2

Related Questions