CODEWITHSUNDEEP
pythonmultithreadingparallel-processingprocessscripting
Biowav
Biowav

Reputation: 311

How to run a python script multiple times simultaneously using python and terminate all when one has finished

Maybe it's a very simple question, but I'm new in concurrency. I want to do a python script to run foo.py 10 times simultaneously with a time limit of 60 sec before automatically abort. The script is a non deterministic algorithm, hence all executions takes different times and one will be finished before the others. Once the first ends, I would like to save the execution time, the output of the algorithm and after that kill the rest of the processes.

I have seen this question run multiple instances of python script simultaneously and it looks very similar, but how can I add time limit and the possibility of when the first one finishes the execution, kills the rest of processes?

Thank you in advance.

Upvotes: 0

Views: 2210

Answers (1)

Asriel
Asriel

Reputation: 190

I'd suggest using the threading lib, because with it you can set threads to daemon threads so that if the main thread exits for whatever reason the other threads are killed. Here's a small example:

#Import the libs...
import threading, time

#Global variables... (List of results.)
results=[]


#The subprocess you want to run several times simultaneously...
def run():
 #We declare results as a global variable.
 global results
 #Do stuff...
 results.append("Hello World! These are my results!")


n=int(input("Welcome user, how much times should I execute run()? "))
#We run the thread n times.
for _ in range(n):
 #Define the thread.    
 t=threading.Thread(target=run)
 #Set the thread to daemon, this means that if the main process exits the threads will be killed.
 t.setDaemon(True)
 #Start the thread.
 t.start()

#Once the threads have started we can execute tha main code.
#We set a timer...
startTime=time.time()
while True:
 #If the timer reaches 60 s we exit from the program.
 if time.time()-startTime>=60:
  print("[ERROR] The script took too long to run!")
  exit()
 #Do stuff on your main thread, if the stuff is complete you can break from the while loop as well. 
 results.append("Main result.")
 break

#When we break from the while loop we print the output.
print("Here are the results: ")
for i in results:
 print(f"-{i}")

This example should solve your problem, but if you wanted to use blocking commands on the main thread the timer would fail, so you'd need to tweak this code a bit. If you wanted to do that move the code from the main thread's loop to a new function (for example def main(): and execute the rest of the threads from a primary thread on main. This example may help you:

def run():
 pass

#Secondary "main" thread.
def main():
 #Start the rest of the threads ( in this case I just start 1).
 localT=threading.Thread(target=run)
 localT.setDaemon(True)
 localT.start()
 #Do stuff.
 pass

#Actual main thread...
t=threading.Thread(target=main)
t.setDaemon(True)
t.start()
#Set up a timer and fetch the results you need with a global list or any other method...
pass

Now, you should avoid global variables at all costs as sometimes they may be a bit buggy, but for some reason the threading lib doesn't allow you to return values from threads, at least i don't know any methods. I think there are other multi-processing libs out there that do let you return values, but I don't know anything about them so I can't explain you anything. Anyways, I hope that this works for you.

-Update: Ok, I was busy writing the code and I didn't read the comments in the post, sorry. You can still use this method but instead of writing code inside the threads, execute another script. You could either import it as a module or actually run it as a script, here's a question that may help you with that: How to run one python file in another file?

Upvotes: 1

Related Questions