CODEWITHSUNDEEP
configurationhadoop
Ahmed
Ahmed

Reputation: 65

The right place for "io.sort.mb" in Hadoop?

I am a bit confused, in the Hadoop cluster setup, in section "Real-World Cluster Configurations", an example is given where properties like io.sort.mb & io.sort.factor goes in core-site.xml. But in the default configuration files these properties appears in mapred-site.xml!!! which one I should follow? Am i free to put it anywhere? If yes, what is the difference between the three configuration file (core, mapred, and hdfs .xml)?

Is there any complete list of all possible properties?

Any help is much appreciated..

EDIT: Trial and error I found io.sort.mb belongs to mapred-site.xml.

Upvotes: 2

Views: 2158

Answers (1)

Praveen Sripati
Praveen Sripati

Reputation: 33545

core-default.html, hdfs-default.html and mapred-default.html have all the properties with their defaults.

According to the 'Hadoop : The Definitive Guide'

core-site.xml has 'Configuration settings for Hadoop Core, such as I/O settings that are common to HDFS and MapReduce.'

hdfs-site.xml has 'Configuration settings for HDFS daemons: the namenode, the secondary namenode, and the datanodes.'

mapred-site.xml had 'Configuration settings for MapReduce daemons: the jobtracker, and the tasktrackers.'

Find more information about the configuration files here.

Upvotes: 1

Related Questions