Hadoop Java - copy file from windows share folder server to HDFS

Question

I want upload multiple file from Windows share folder server (e.g. //server_name/folder/) to my HDFS using Java

list of methods I have tried

• org.apache.hadoop.fs.FileUtil set input path = //server_name/folder/ it says java.io.FileNotFoundException: File //server_name/folder/ does not exist

• FileSystem.copyFromLocalFile (i think this is from local hadoop server to hdfs server)

• IOUtils.copyBytes same as fileUtil >> file does not exist

• a simple File.renameTo same as fileUtil >> file does not exist

  String source_path = "\\server_name\folder\xxx.txt";

  String hdfs_path = "hdfs://HADOOP_SERVER_NAME:Port/myfile/xxx.txt";

  File srcFile = new File(source_path);

  File dstFile = new File(hdfs_path);

  srcFile.renameTo(dstFile);

Do I need to create FTP or How about using FTPFileSystem?

Or anyone have better solution Or Sample Code

thank you

Answer 1

FileSystem has copyFromLocal method:

import org.apache.hadoop.conf.Configuration; 
import org.apache.hadoop.fs.FileSystem;

Configuration configuration = new Configuration();
configuration.set("fs.defaultFS", "hdfs://abc:9000");

FileSystem fs= FileSystem.get(configuration);
fs.copyFromLocalFile(new Path("/source/directory/"), 
  new Path("/user/hadoop/dir"));