CODEWITHSUNDEEP
pythonmultithreadingrace-condition
Yen Hung Liu
Yen Hung Liu

Reputation: 180

Signal or mutex in Python threading

I wrote a Python script which use GCC and dockross to cross build an application on Windows and Linux. Because I want to speed up the time spending on building code, I use threading module to implement all four building processes (Linux and Windows, 32 and 64 bit).

But I found the two GCC building objects (for Linux 32 bit and 64 bit) are in a race condition. If I want to do the two buildings at the same time, some errors will occur, and the same situation happened on the dockcross building process.

The two dockcross building objects are in a race condition.

Are there any function or module I can use in Python for me to implement the two threads as a coupled thread, when one thread is finished, it will signal to its coupled thread to start?

Like the code below, I want the worker[0] and worker[1] have a signal mechanism and also between worker[2] and worker[3].

def main():

linux32_builder = builder(
    "linux32", "make  CFLAGS=-m32 all", "./", "./", "/root/crossCompile/releaseFolder/")

linux64_builder = builder(
    "linux64", "make   all", "./", "./", "/root/crossCompile/releaseFolder/")

win64_builder = builder(
    "win64", "dockcross-windows-x64 make  all", "./", "./", "/root/crossCompile/releaseFolder/")
win32_builder = builder(
    "win32", "dockcross-windows-x86 make  all", "./", "./", "/root/crossCompile/releaseFolder/")

# linux32_builder.copySourceCodeFromShare()
Worker = []
Worker.append(Make_Thread(1, linux32_builder))
Worker.append(Make_Thread(2, linux64_builder))
Worker.append(Make_Thread(3, win64_builder))
Worker.append(Make_Thread(4, win32_builder))

for i in Worker:
    i.start()
    i.join()

Upvotes: 0

Views: 478

Answers (1)

Cris P
Cris P

Reputation: 433

In Python, thread.join() blocks until the thread terminates. I believe your code sample does not run the threads in parallel, instead waiting for each thread to terminate before starting the next. You may want to verify that each thread does not terminate before the actual build process completes, otherwise you may still have race conditions. (I presume the build uses external processes.)

Assuming that your various _builder objects are functions and that Make_Thread uses them as main thread functions, I would suggest using sequential execution rather than threads to get the effect you desire.

The code could look something like this:

linux = Make_Thread(1, lambda: linux32_builder(), linux64_builder())
windows = Make_Thread(2, lambda: win32_builder(), win64_builder())
linux.start()
windows.start()
linux.join()
windows.join()

(I am guessing at the numeric arguments to Make_Thread.)

The lambda expressions actually return tuples, but it is guaranteed that the builders will be called in order from left to right.

Alternatively, you could make two threads in addition to the four shown in your code, and these two threads could each control two of the other threads. The Linux builder thread main function would look something like:

def linux():
  Worker[0].start()
  Worker[1].start()
  Worker[0].join()
  Worker[1].join()

and the the Windows builder thread main function would look like:

def windows():
  Worker[2].start()
  Worker[3].start()
  Worker[2].join()
  Worker[3].join()

You would then start each of these two controlling threads and finally join them. This approach is more complex, with more code and more threads.

Upvotes: 1

Related Questions