CODEWITHSUNDEEP
javaamazon-web-servicesamazon-s3mapreduceemr
User23890
User23890

Reputation: 609

Would resizing a running EMR cluster speeds up the execution of the existing running job

I am running an EMR cluster with 25 nodes and a job is running on for continuously 15-20 minutes. I realized that it would take another 30-40 minutes so I added 10 more task nodes in the cluster. But I didn't see any improvement.

Would adding these extra task nodes increase the concurrent mappers execution in current job or the extra nodes will be used for only the jobs initiated after resizing?

Upvotes: 1

Views: 654

Answers (1)

ChristopherB
ChristopherB

Reputation: 2068

It depends on how the MapReduce job is structured. If there was tasks pending then the additional nodes will be used. If the job determined its number of tasks based on original size of the cluster then additional nodes will not make a difference until new job submission.

Upvotes: 1

Related Questions