CODEWITHSUNDEEP
sas
Matt
Matt

Reputation: 171

Parallel processing in SAS, waiting for free sessions

Say I have 2 sessions signed on:

signon slave1;
signon slave2;

Say I have 10 jobs executed by

%execute_job( parameter= 1);
%execute_job( parameter= 2);
%execute_job( parameter= 3);
.
.
.
%execute_job( parameter= 10);

What do I need to add into %execute_job so it will keep waiting for a free session and submit the job to the first free session it finds?

I tried the macvar stuff to no avail.

Upvotes: 2

Views: 465

Answers (2)

vasja
vasja

Reputation: 4792

Consider this: I use cmacvar option on signon to dedicate a local macro var for remote session status. In syslput, I created remote my_id macro var on both slaves so that they know "who they are".

Then I have %job macro receiving ID of slave to run some code on. For real world there should also be some parametor of job to run. Here I just simulate shorter job1 and longer job2.

In %run_jobs I do some looping to try to run the slaves several times. The main point is in WAITFOR _ANY_ statement to do the waiting for you.

signon slave1 sascmd="!sascmd" cmacvar=slave1; 
signon slave2 sascmd="!sascmd" cmacvar=slave2; 

%syslput my_id=1 /remote=slave1;
%syslput my_id=2 /remote=slave2;

rsubmit slave1;
options nomprint nosource nonotes;
endrsubmit;
rsubmit slave2;
options nomprint nosource nonotes;
endrsubmit;
/*
my_id is remote
slave_id is local
*/;
%macro job(slave_id);
rsubmit slave&slave_id. wait=no;
    data _null_;
         %if &slave_id eq 2 %then %do;
            x = sleep(2);
         %end;
         x = sleep(4);
         s = put(DATETIME(), DATETIME.) || " at &my_id.!";
        putlog s;
    run;
endrsubmit;
%mend;


%macro run_jobs;
    %do i=1 %to 4;
        %if &slave1 eq 0 %then %do;
            %put Slave1 starting;
            %job(1)
        %end;
        %else %put Slave1 working;
        %if &slave2 eq 0 %then %do;
            %put Slave2 starting;
            %job(2)
            %put Slave2: &slave2;
        %end;
        %else %put Slave2 working;
        WAITFOR _any_ slave1 slave2;
        *LISTTASK _ALL_ ;
        %put at end of loop &i:;
        %put Slave1: &slave1;
        %put Slave2: &slave2;
    %end;
%mend;


options nomprint nosource nonotes;
%run_jobs

signoff slave1;
signoff slave2;

Upvotes: 3

Matt
Matt

Reputation: 171

Alright, I think I got something:

options sascmd="!sascmd";


%macro job(value);

    %local slave_id;

    %let slave_id = 1;

    %do  %while(&&slave&slave_id.);
        %put Slave&slave_id. is busy and cannot be assigned job;

        %if &slave_id. < 2  %then
            %let slave_id = %eval(&slave_id. + 1);
        %else 
            %let slave_id = 1;
    %end;

    %put Slave&slave_id. is free and is assigned job;

    %syslput _user_ /remote= slave&slave_id.;

    rsubmit slave&slave_id. macvar=slave&slave_id. wait=no;


        data _null_;

            file print;
            put "&value. by &slave_id.!";

        run;

    endrsubmit;

%mend job;

signon slave1 macvar=slave1;
signon slave2 macvar=slave2;

%job(6);

%job(7);

%job(8);

signoff slave1;
signoff slave2;

Upvotes: 1

Related Questions