CODEWITHSUNDEEP
hadoophadoop-yarn
user5228393
user5228393

Reputation:

Role of the ApplicationManager in YARN

enter image description here

I am reading a book and the above picture is described as the ResourceManager in YARN.

They state:

  1. Admin and Client Service is responsible "for client interactions such such as a job request submission, start, restart, and so on"
  2. "ApplicationMaster Launcher launches the container for the ApplicationMaster on job submission from client."

Then they contradict what they said above with the below statement:

The ApplicationManager is responsible for accepting the client’s job submission requests, negotiating the containers to execute the applications specific to the ApplicationMaster, and providing the services to restart the ApplicationMaster on failure.

Question:

I thought the role of the of the Admin and Client Service was to accept the client job submission and not the ApplicationManager role? So essentially does the ApplicationManager get the job from the Admin and Client service and uses the ApplicationMaster Launcher to actually launch the ApplicationMaster with the resources specified in the job submission?

Upvotes: 1

Views: 1547

Answers (2)

Ravindra babu
Ravindra babu

Reputation: 38910

You can find more details about ApplicationsManager in YARN official documentation page.

enter image description here

The idea is to have a global ResourceManager (RM) and per-application ApplicationMaster (AM)

The per-application ApplicationMaster is, in effect, a framework specific library and is tasked with negotiating resources from the ResourceManager and working with the NodeManager(s) to execute and monitor the tasks.

The ResourceManager is the ultimate authority that arbitrates resources among all the applications in the system.

The ResourceManager has two main components: Scheduler and ApplicationsManager.

The Scheduler is responsible for allocating resources to the various running applications subject to familiar constraints of capacities, queues etc. The Scheduler does not performs monitoring or tracking of status for the application.

The ApplicationsManager is responsible for

  1. Accepting job-submissions
  2. Negotiating the first container for executing the application specific ApplicationMaster
  3. And provides the service for restarting the ApplicationMaster container on failure.

The per-application ApplicationMaster has the responsibility of negotiating appropriate resource containers from the Scheduler, tracking their status and monitoring for progress.

Regarding your query

When the ApplicationManager retrieves the job from the Admin and Client Service, does the job contain the resources that is required to launch an ApplicationMaster? Like how does the ResourceManager know how much resources the ApplicationMaster needs?

ResoruceManager ( ApplicationsManager+Scheduler) in collaboration with ApplicationsMaster make sure that resources are allocated for the client request and they perform the roles as explained above.

Upvotes: 2

siddhartha jain
siddhartha jain

Reputation: 1006

Yes, your assumption is correct application manager get the job from admin and Client service. ApplicationsManager is responsible for maintaining a collection of submitted applications or jobs. After application submission, it first validates the application’s specifications and rejects any application that requests unsatisfiable resources for it.Once the application manager is satisfied then application master launcher, launches application master. The application master is responsible for negotiating resources from Resource Manager and working with node manager to execute and monitor the container and their resource consumption. one important thing is that each job have its own application master. Which is not applicable for application manager.

You can visit this link http://javacrunch.in/Yarn.jsp if you want to see the communication flow within Yarn.

Hope this solve your query.

Upvotes: 0

Related Questions