CODEWITHSUNDEEP
mesosmesosphere
Javi Carnero
Javi Carnero

Reputation: 467

Modify mesos agent to add custom resources that change dynamically

I'm developing a new mesos-slurm framework where jobs from outside mesos can also be pushed to slurm queues.

UPDATE to explain better the problem: The mesos agent has a slurm workload manager installed in the same computer that orchestrates jobs in a HPC. This Slurm receive jobs either from the mesos executor as from other methods (for example third-party users sending jobs directly to slurm through ssh).

Therefore I'd like the agent could know, before sending offers to mesos, the state of the slurm queues (number of jobs running and waiting to run), and offer resources accordingly. This can not be achieved only by knowing the tasks accepted by the executor, as other resources of the HPC could have been taken by third-party users using slurm directly.

In other words what I'd like to do is customize the way the agent know the resources available to offer, to take into account the current state of Slurm queues.

UPDATE 2 Diagram to explain the situation: enter image description here

As you can see in the diagram, the way the agent creates offers (in red) is what I'd like to customise so the agent would take into account the state of Slurm (this can be read by the executor or the agent) before generate them.

Is this possible? If positive, how could be achieved?

Thanks in advance.

Upvotes: 2

Views: 289

Answers (1)

janisz
janisz

Reputation: 6371

I think this design is broken. If you allow jobs to be started without Mesos control you lost control of resources.

  • If you can prepare simple framework that will proxy users request to Mesos: you can use custom resources and get current status in offers.

  • If you can't change your design and you need to start some jobs outside of Mesos: you can use oversubscription mechanism with combination with custom resources. Then your QoS will update resources according to current usage and you get usage in offer.

Upvotes: 1

Related Questions