CODEWITHSUNDEEP
hadoop
user1050619
user1050619

Reputation: 20916

Cluster configuration and hdfs

Im trying to configure my cluster by following this tutorial - 

https://developer.yahoo.com/hadoop/tutorial/module2.html


<configuration>
  <property>
    <name>fs.default.name</name>

    <value>hdfs://192.168.71.128:9000</value>
  </property>
  <property>
    <name>dfs.data.dir</name>

    <value>/home/hadoop-user/hdfs/data</value>
  </property>
  <property>
    <name>dfs.name.dir</name>

    <value>/home/hadoop-user/hdfs/name</value>
  </property>
</configuration>

I have also copied a local file to /user/prema/ using the below commands

hadoop-user@hadoop-desk:~/hadoop$ bin/hadoop dfs -put /home/hadoop-user/googlebooks-eng-all-1gram-20120701-0 /user/prema
hadoop-user@hadoop-desk:~/hadoop$ bin/hadoop dfs -ls /user/prema
Found 1 items
-rw-r--r--   1 hadoop-user supergroup  192403080 2014-11-19 02:43 /user/prema

Now, I'm confused. I have datafiles here- /user/prema but the data node in the cluster config points to this - /home/hadoop-user/hdfs/data..How does it get related?

Upvotes: 0

Views: 69

Answers (1)

Lauri Peltonen
Lauri Peltonen

Reputation: 1542

The /user/prema is a folder within HDFS. The folder /home/hadoop-user/hdfs/data is a folder within the regular filesystem.

The regular filesystem folder is the place where HDFS stores its data. So when you read data from HDFS, it actually goes to the physical regular filesystem folder to read the data. You should never need to touch this data as its format is not very user-friendly - the HDFS takes care of data manipulation for you.

Upvotes: 1

Related Questions