CODEWITHSUNDEEP
pythonmultithreading
tsionyx
tsionyx

Reputation: 1669

Which mechanism to use to properly release resource acquired by many threads

Suppose I have two functions presented below. The first one (use_stuff) can be used by many threads to do some computation, while the second one (clean_stuff_cache) can be called occasionally in separate thread when we need to update cache. Which synchronization mechanism (I call it counter here) should I use to ensure that the clean_stuff_cache will block until all the threads finish the use_stuff routine.

stuff_cache = dict()
counter = ????

def use_stuff(id):
   counter.acquire()

   if id not in stuff_cache:
      stuff_cache[id] = get_stuff(id)
   # do some computation using stuff
   do_stuff(stuff_cache[id])

   counter.release()

def clean_stuff_cache(id):
   counter.wait_until_zero()
   # drop all the stuff being used
   del stuff[id]

Upvotes: 1

Views: 84

Answers (1)

Thomas Moreau
Thomas Moreau

Reputation: 4467

You can use a Semaphore object here combine with a Lock. The semaphore permits to have a synchronized counter that block if you try to acquire if when it is 0.
So if counter = Semaphore(0), and lock = Lock() then:

def use_stuff(id):
    # Increment number of running workers if your are not cleaning the cache
    with lock:
        counter.release()
    ..... do stuff...
    # decrement number of running workers
    counter.acquire()

def clean_stuff():
    # counter.acquire return False if there is no worker running
    while True:
        lock.acquire()
        if counter.acquire(False):
            counter.release()
            lock.release()
        else:
            break

        # delay next loop so the check is only occasional
        time.sleep(.01)

    # here, no worker is running and the lock prevents new worker
    # from starting so you can cleanup the cache...
    ... do clean up
    lock.release()

Upvotes: 1

Related Questions