CODEWITHSUNDEEP
pythonconcurrencytimerscheduler
Jason Donnald
Jason Donnald

Reputation: 2316

issue in trying to execute certain number of python scripts at certain intervals

I am trying to execute certain number of python scripts at certain intervals. Each script takes a lot of time to execute and hence I do not want to waste time in waiting to run them sequentially. I tired this code but it is not executing them simultaneously and is executing them one by one:

Main_file.py

import time
def func(argument):

    print 'Starting the execution for argument:',argument
    execfile('test_'+argument+'.py')


if __name__ == '__main__':

    arg = ['01','02','03','04','05']

    for val in arg:
        func(val)
        time.sleep(60)

What I want is to kick off by starting the executing of first file(test_01.py). This will keep on executing for some time. After 1 minute has passed I want to start the simultaneous execution of second file (test_02.py). This will also keep on executing for some time. Like this I want to start the executing of all the scripts after gaps of 1 minute.

With the above code, I notice that the execution is happening one after other file and not simultaneously as the print statements which are there in these files appear one after the other and not mixed up.

How can I achieve above needed functionality?

Upvotes: 0

Views: 110

Answers (2)

Schiem
Schiem

Reputation: 589

Using python 2.7 on my computer, the following seems to work with small python scripts as test_01.py, test_02.py, etc. when threading with the following code:

import time
import thread

def func(argument):
    print('Starting the execution for argument:',argument)
    execfile('test_'+argument+'.py')


if __name__ == '__main__':

    arg = ['01','02','03']

    for val in arg:
        thread.start_new_thread(func, (val,))
        time.sleep(10)

However, you indicated that you kept getting a memory exception error. This is likely due to your scripts using more stack memory than was allocated to them, as each thread is allocated 8 kb by default (on Linux). You could attempt to give them more memory by calling

thread.stack_size([size])

which is outlined here: https://docs.python.org/2/library/thread.html

Without knowing the number of threads that you're attempting to create or how memory intensive they are, it's difficult to if a better solution should be sought. Since you seem to be looking into executing multiple scripts essentially independently of one another (no shared data), you could also look into the Multiprocessing module here:

https://docs.python.org/2/library/multiprocessing.html

Upvotes: 2

Donkyhotay
Donkyhotay

Reputation: 81

If you need them to run parallel you will need to look into threading. Take a look at https://docs.python.org/3/library/threading.html or https://docs.python.org/2/library/threading.html depending on the version of python you are using.

Upvotes: 0

Related Questions