CODEWITHSUNDEEP
hadoopmapreduce
Chinmaya Barik
Chinmaya Barik

Reputation: 1

Output path for shuffle and sort phase

Where is actually the output of shuffle & sort phase of a MR programming stored is it stored in local disk or in HDFS?

Upvotes: 0

Views: 394

Answers (2)

HbnKing
HbnKing

Reputation: 1882

Of course it stored in local disk because there is no need to duplicate the temporary file. if the map task fail it will retry . here is the default setting in mapred-site.xml

mapreduce.cluster.local.dir ${hadoop.tmp.dir}/mapred/local

Upvotes: 0

dbustosp
dbustosp

Reputation: 4458

The data is stored in the Local File System (Not HDFS). This is tipically a temporary directory which is cleaned after Hadoop job completes and the configuration is in core-site.xml.

Upvotes: 1

Related Questions