CODEWITHSUNDEEP
hadoopmapreduce
kee
kee

Reputation: 11629

How to change io.sort.mb per job?

I am wondering if I can change the value of io.sort.mb per job? I know you can set a value for the parameter in mapred-site.xml but I want to use different values in different jobs programatically. I tried conf.setInt("io.sort.mb", someValue) but it doesn't seem to work. JVM setting has enough memory (like 2.25GB) and there is no other jobs running.

Upvotes: 2

Views: 1163

Answers (1)

Sean Owen
Sean Owen

Reputation: 66881

Sure, just set it in the Configuration (JobConf in earlier versions) before you submit. It does work; I used it in Mahout. Make sure you are setting conf onto your Job after you set the value and before you submit. Make sure you are setting the right conf too! Maybe you can say how you know it isn't working.

Upvotes: 2

Related Questions