CODEWITHSUNDEEP
pythonmultithreading
Rtrain
Rtrain

Reputation: 55

Script is printing in staggered order - possible thread safety issue

I'm trying to print urls in a list with num_thread threads. Here is the code, with alphabets in place of urls:

import requests
import threading
import time

def url_res(url, lock):
    with lock:
        print(url)

def q_9(url_list, num_threads):
    
    thread_list = []
    
    for url in url_list:
        lock = threading.Lock()
        t = threading.Thread(target = url_res, args = [url, lock])
        thread_list += [t]
        t.start()
        for i in thread_list:
            if (i.isAlive() == False):
                thread_list.remove(i)
        while len(thread_list) > num_threads:
            thread_list[0].join()
            thread_list.remove(thread_list[0])

q_9(["abc", "def", "ghi", "jkl", "mno", "pqr"], 3)

However, I don't get uniform outputs as you'd expect in thread-safe functions (if my understanding is correct):

abc
def
ghijkl

mnopqr

Or:

abc
defghi

jkl
mno
pqr

Or:

abcdef

ghi
jkl
mnopqr

Etc. Is it actually a problem with the locking/thread safety? Or is it something else?

Upvotes: 1

Views: 93

Answers (1)

Cihan
Cihan

Reputation: 2307

The problem here is that you are creating a separate lock for each thread. The whole point with using locks is that you want threads to synchronize by forcing them to acquire the same lock.

You can fix it by lifting locks scope up, and making threads share a single lock:

import requests
import threading
import time


def url_res(url, lock):
    with lock:
        print(url)

def q_9(url_list, num_threads):
    
    thread_list = []
    lock = threading.Lock() # share a single lock
    
    for url in url_list:
        t = threading.Thread(target = url_res, args = [url, lock])
        thread_list += [t]
        t.start()
        for i in thread_list:
            if (i.isAlive() == False):
                thread_list.remove(i)
        while len(thread_list) > num_threads:
            thread_list[0].join()
            thread_list.remove(thread_list[0])

q_9(["abc", "def", "ghi", "jkl", "mno", "pqr"], 3)

Output:

abc
def
ghi
jkl
mno
pqr

Upvotes: 1

Related Questions