CODEWITHSUNDEEP
pythonmultithreadingsynchronize
Richard Dorman
Richard Dorman

Reputation: 24150

Synchronising multiple threads in python

I have a problem where I need x threads to wait until they have all reached a synchronization point. My solution uses the synchronise method below which is called by each threaded function when they need to synchronise.

Is there a better way to do this?

thread_count = 0
semaphore = threading.Semaphore()
event = threading.Event()

def synchronise(count):
    """ All calls to this method will block until the last (count) call is made """
    with semaphore:
        thread_count += 1
        if thread_count == count:
            event.set()

    event.wait()

def threaded_function():
    # Do something

    # Block until 4 threads have reached this point
    synchronise(4)

    # Continue doing something else

Upvotes: 6

Views: 9093

Answers (3)

testworks
testworks

Reputation: 396

Note that Barrier has been implemented as of Python 3.2

Example of using barriers:

from threading import Barrier, Thread

def get_votes(site):
    ballots = conduct_election(site)
    all_polls_closed.wait()        # do not count until all polls are closed
    totals = summarize(ballots)
    publish(site, totals)

all_polls_closed = Barrier(len(sites))
for site in sites:
    Thread(target=get_votes, args=(site,)).start()

Upvotes: 4

S.Lott
S.Lott

Reputation: 392060

There are many ways to synchronize threads. Many.

In addition to synchronize, you can do things like the following.

  1. Break your tasks into two steps around the synchronization point. Start threads doing the pre-sync step. Then use "join" to wait until all threads finish step 1. Start new threads doing the post-sync step. I prefer this, over synchronize.

  2. Create a queue; acquire a synchronization lock. Start all threads. Each thread puts an entry in the queue and waits on the synchronization lock. The "main" thread sits in a loop dequeueing items from the queue. When all threads have put an item in the queue, the "main" thread releases the synchronization lock. All other threads are now free to run again.

There are a number of interprocess communication (IPC) techniques -- all of which can be used for thread synchronization.

Upvotes: 2

user9876
user9876

Reputation: 11102

The functionality you want is called a "barrier". (Unfortunately that term has 2 meanings when talking about threading. So if you Google it, just ignore articles that talk about "memory barriers" - that's a very different thing).

Your code looks quite reasonable - it's simple and safe.

I couldn't find any "standard" implementations of barriers for Python, so I suggest you keep using your code.

Upvotes: 2

Related Questions