CODEWITHSUNDEEP
hadoopmemorymemory-managementjvmhadoop-yarn
Lin Ma
Lin Ma

Reputation: 10149

Hadoop parameter mapreduce.map.memory.mb and mapreduce.map.java.opts

Wondering what are the relationship between the two parameters (mapreduce.map.memory.mb and mapreduce.map.java.opts), and sometimes I see people setup mapreduce.map.memory.mb to be 2048 and mapreduce.map.java.opts to be -Xmx4G -Xms4G -server, wondering what are the upper limit bound in this case? Thanks.

regards, Lin

Upvotes: 0

Views: 3941

Answers (1)

banjara
banjara

Reputation: 3890

mapreduce.map.memory.mb is resource limit for maps.

mapreduce.map.java.opts is heap-size for child jvms of maps.

The Hadoop setting is more of a resource enforcement/controlling one and the Java is more of a resource configuration one.

The Java heap settings should be smaller than the Hadoop container memory limit because we need reserve memory for Java code. Usually, it is recommended to reserve 20% memory for code.

Upvotes: 3

Related Questions