CODEWITHSUNDEEP
pythonmultithreadingio
Brad
Brad

Reputation: 6342

will non-daemon python threads exit after they're done with their task?

Context: I've got a simple python script that writes a bunch of files to s3 (~70-100) every few seconds. Because its I/O bound, I wanted to thread the write process so the script performs better. I'm using threading to build my threads.

Question: Since my threads are a) non-daemons and b) they only have 1 task to perform (write a file), if I loop over my list of threads and call .join() will they finish their task and exit gracefully? Do I even need to call join() here or will they just exit when they're done? I believe join() is the way to go here but since i'm very new to python, I don't know what I don't know....

Here's some simplified code for reference: 

buildOutput() #calls the section below 
for thread in threads:
   thread.join()
time.sleep(60)

calls: 

  for item in out[fileRoot]: 
        #write individiual files
        key = findKey(item, FILE_KEY)
        full_key = FILE_PATH + str(key) + FILE_TYPE
        t = FileWriter(item, full_key)
        t.start()
        threads.append(t) #global threads list for the script

where FileWriter is the class that does the writing.

Upvotes: 1

Views: 200

Answers (1)

sgun
sgun

Reputation: 899

Join makes sure that the main thread will wait until joined thread finishes its task. There is a good ascii art given here.

So you'd better use join when your child threads are performing I/O to prevent any unexpected behavior.

Upvotes: 1

Related Questions