CODEWITHSUNDEEP
javahadoophdfs
Rurien
Rurien

Reputation: 369

Does java hdfs client need to have core-site.xml and hdfs-site.xml for connection?

I downloaded files from Hadoop server with HttpClient.

But other people said it's not good for security because it doesn't need to input user ID. So I have to change it to webhdfs or hdfs.

I have to connect with Hadoop and it's not mine.

So I don't know all of its configurations.

They told me the only username, some IP addresses, and ports.

But hdfs client in java program needs many configurations.

Does hdfs client need core-site.xml and hdfs-site.xml for its configuration?

or do I have some other way to resolve it?

Upvotes: 1

Views: 1567

Answers (1)

Victor Kim
Victor Kim

Reputation: 1767

In order to communicate with HDFS properly a client needs to know configuration parameters. And this is what those two files are all about and client needs configs from both files. As it stated here:

The core-site.xml file informs Hadoop daemon where NameNode runs in the cluster. It contains the configuration settings for Hadoop Core such as I/O settings that are common to HDFS and MapReduce.

The hdfs-site.xml file contains the configuration settings for HDFS daemons; the NameNode, the Secondary NameNode, and the DataNodes.

But in Java code, all configs (no matter from which file) are being carried by org.apache.hadoop.conf.Configuration. So, you can place everything in one Configuration object.

Hope this answers your question.

Upvotes: 3

Related Questions