CODEWITHSUNDEEP
cluster-computingsungridengineqsub
The Zoo
The Zoo

Reputation: 55

sun grid engine qsub to all nodes

I have a master and two nodes. They are install with SGN. And I have a shell script ready on all the nodes as well. Now I want to use a qsub to submit the job on all my nodes. I used:

qsub -V -b n -cwd /root/remotescript.sh

but it seems that only one node is doing the job. I am wondering how do I submit jobs for all nodes. What would the command be. My reference is this enter link description here

Upvotes: 0

Views: 1016

Answers (2)

Vince
Vince

Reputation: 3395

Finch_Powers answer is good for describing how SGE allocates resources. So, I'll elaborate below on specifics of you question, which may be why you are not getting the desired outcome.

You mention launching remote script via:

qsub -V -b n -cwd /root/remotescript.sh

Also, you mention again that these scripts are located on the nodes:

"And I have a shell script ready on all the nodes as well"

This is not how SGE is designed to work, although it can do this. Typical usage is to have same single (or multiple) scripts accessible to all nodes via network mounted storage on the execution nodes and let SGE decide which nodes to run the script on.

To run remote code, you may be better served using plain SSH.

Upvotes: 1

Finch_Powers
Finch_Powers

Reputation: 3106

SGE is meant to dispatch jobs to worker nodes. In your example, you create one job so one node will run it. If you want to run a job on each of your node, you need to submit more than one job. If you want to target nodes you probably should use something closer to

qsub -V -b n -cwd -l hostname=node001 /root/remotescript.sh
qsub -V -b n -cwd -l hostname=node002 /root/remotescript.sh

The "-l hostname=*" parameter will require a specific host to run the job.

What are you trying to do? The general use case of using a grid engine is to let the scheduler dispatch the jobs so you don't have to use the "-l hostname=*" parameter. So technically you should just submit a bunch of jobs to SGE and let it dispatch it with the nodes availability.

Upvotes: 2

Related Questions